Backdoor corporate sabotage with DNS

I’m not really certain how common corporate sabotage is. Sure there are DOS attacks daily on this or that network or this or that server but what percentage of those are script kiddies and what percentage are well thought out planned attacks designed to cripple a competitor even if only for a short time. Typically DOS attacks are dealt with by Server and Network Admins adding black holes to offending networks. Recently while doing some research I stumbled on what seems to be a neglected DNS attack. One that the target may not become aware of until the next billing cycle or if carried out methodically months.

Ultradns and Dnsmadeeasy are two leading hosted DNS providers. The model is simple. You pay to have your dns hosted on their network and servers. They ensure DNS propagation between their servers is fast and they have the capacity to protect against DOS attacks. Typically you get some base package of queries per month. For example the Business Membership is 59.95/yr and you get 10 million queries/month. That is a lot of queries when you consider that many queries for your domain will be served by caching servers. And overage charges are minimal at $6.00/ 1 million queries (if you don’t purchase blocks ahead of time). I was thinking boy I hope there is some kind of throttling in place to prevent some unsavory competitor from looping a dig against their name servers for my domain. So on a whim i looked around and actually found a domain (on the first try I might add) which uses Dnsmadeeasy. Oh in case you were wondering how I found out, I just did a whois and looked at the authoritative name servers and wow ns0.dnsmadeeasy.com was listed. So I ran a quick loop for 100 lookups on the domain.

time for i in $(seq 100);do dig redacted.tld @ns0.dnsmadeeasy.com;done

While I expected to get off a few lookups and then just wait for some throttle timeout to shut me down I was supprised to get all 100 lookups done in 11 seconds, subsequent tests showed similar times mostly faster. So conservativly say you can do 10 lookups a second. If my math serves me correct you can do 10 million lookups in just about 5 hours. After that you have broken the 10 million limit for the month. Holding steady at the same rate thats 864000 queries in 24 hours and 25920000 in 30 days. Yeah so not a bank breaker at $6/ million but this was from a single PC and I doubt network was the bottle neck. A distributed attack could end up costing a company thousands upon thousands. Refusal to pay could result in DNS being shut off, and effectively creating a DOS. For fun I tried 100 lookups against Ultradns for some of their banner customers and also received no throttling. Still a bit surprised at this seemingly overlooked hole I called Dnsmadeeasy and asked the sales department what protections were in place to prevent or mitigate malicious lookups. His response was do you mean DOS? When I explained the issue he said we can not block that, as we do not know if there are 1000 people behind your company firewall that are really interested in that website.

It does not seem unreasonable to provide a throttling mechanism. Oh you queried 10 times in the last 2 seconds? I think I will block you for 5 mintues. Happens again within x time increase block time to 10 minutes and so on. So who wants to loop while true dig amazon @udns1.ultradns.net for a month or so and see what happens. Will they report being hacked? Will the cops bust down your door? Will amazon just eat the cost (probably). But why not just provide simple throttling for obviously either misconfigured or malicious lookups?

10 Comments

  • DNS Monster Windows XP Firefox 3.0 wrote:

    I work for Ultradns (actually known as ‘NeuStar Ultra Services’). It is VERY common for companies to make no changes to their site, no changes to their DNS (ie – their TTL…which they can set as low as “0” or “300 seconds”…depending on their account) but their query count will suddenly quintuple for one or two months with no perceivable reason. They are liable for the overage.

    I have spoken to many customers that will look for a reason they have suddenly received a $3,000 bill (on a monthly service of $50 to $150) for overages on queries alone and been forced by the Director of Support , Ray McKenzie to tell them “sorry, but you are liable…pay the bill or have your service cancelled. We cannot provide a log for this period, you must request that logging be started for a period and thats all we can give you”.

  • Greg Windows XP Firefox 3.0 wrote:

    Why would you want DNS providers to throttle DNS requests? The whole point behind an enterprise DNS provider is to insure 100% results to legitimate queries. It sounds like you believe there is some kind of simple mechanism that will determine if your 100 queries are “legitimate” or not, however there is not. The servers could filter your queries and determine if they were duplicates and limit or block the source IP based on that, but the processing of this logic for all queries hitting the DNS server would more than likely cost more in resources (and thus raise pricing by the provider) and would render it useless for the purposes of saving you money on your DNS bill.

    Other than duplicate checks, what other logic would you propose in determining a legitimate DNS query? Consider that the popularity of domains will vary and that many ISPs use the same resolving nameservers for large geographic areas. These nameservers would be very likely blocked if the throttling concept you propose were in place, as they reguarly make queries for tens of thousands of users. Beyond this, DoS/DDoS mitigation would protect against high level attacks to nameservers.

    The issue you describe is no different than hosting providers charging users for additional bandwidth for other types of traffic, as there is no simple way to prevent someone from spoofing TCP SYN to a server, which would ultimately cause a rise in bandwidth utilization for that customer. Its simply a cost of providing the service on the public Internet. DoS/DDoS mitigation can be utilized to prevent this in some manner, but saving a few extra dollars by implementing complex filtering mechanisms isn’t worth the false positives its likely to cause in the end.

    Also, I was wondering if you happen to use spell check because even the title of your article has a number of incorrectly spelled words.

  • Michael James Windows XP Internet Explorer 7.0 wrote:

    I don’t understand your point here…. And honestly it shows very little knowledge of the industry or of any financial business.
    If someone calls your cell phone and you pick up, even if it’s a wrong number, don’t you get charged?
    What happens if you had a 1-800 service and people call you by accident, don’t you get charged?
    Don’t you think UltraDNS and DNS Made Easy get’s charged for the bandwidth that these requests take up? Should they just not charge you for them because you don’t want them?
    If you host email somewhere and people send you a lot of spam then you could get charged overages. This is called “hosting”.

    On top of your lack of understanding of the business and finances you also have no technical expertise as well….
    What happens if you have a popular domain and you are getting a lot of queries?
    How would anyone know if the query is legitimate or not?

    Please do more research on the area and then come back with a reasonable aurgument to support your case. As it stands right now your whole argument is just foolish.

  • Greg perhaps you do not understand dns caching. My isp has nameservers that almost every user uses. Im sure a large portion of them look up google.com at least once a day. The first person that queries against my isps nameserver for google will cause my isps name server to cace that result for the ttl of the record, which according to my lookups google currently has set to 300s or 5 minutes. So if any other isp user queries my nameserver (witin the ttl) for google my nameserver will answer without going back to googles nameservers for an answer. That is proper behavior. I say that throttling requests may be a good idea because other name servers should be following the ttl between requests. And I really dont see how having a trigger on a request to drop in a firewall rule for a period of time would be that expensive in terms of cputime. You make it sound like those enterprise DNS services are running all of their dns servers at full capacity and some simple ttl enforcement would be the fly that broke the camels back.

    Greg and Michael James, I didn’t say the Enterprise services should not charge for overage. I said there should be a mechanism to allow the customer to controll the overages. If someone calls my cell phone I can choose to answer the call or not and therefor choose weather to be charged for the wrong number call. I can also choose to blackhole traffic from a given ip address on my firewall to stop myself from incurring overage charges for bandwidth.

    I dont think its hard to understand, provide the ability to enforce TTL between queries, some type of x queries inside ttl time from same source address before black holing for x time.

    Right now Michael, your whole argument is your service provider should provide no mechanism fo you to prevent malicious query attacks.

    DNS Monster claims to work for NeuStar and says that its not uncommon for these overcharge situations to arise.

    Looking forward to your deep insight into the industry and financial business.

    Oh and Greg, I have a post addressing my misspell laden posts. I don’t write for profit, and I am surprised when anyone does decide to read anything I write. Its my blog and Ill misspell if I want to. If you don’t like it don’t read.

  • Wow, Michael, a bit harsh, eh?

    Greg and Michael, this sort of attack is a valid concern, and should be considered tantamount to a DOS, since if you can’t pay the bill, your service is gone.

    Every good IPS should have the ability to throttle service requests, especially ones that originate from the same IP. As CMDLN stated, there’s no valid reason for the same IP to make hundreds of DNS requests a day, let alone in a second. He’s right, TTL should be respected, and even in clients where it isn’t, I’ve never even seen a setting for force recaching in shorter than 300 seconds (5 minutes).

    The most extreme case I can think of, a slashdotting, with fark, reddit, and digg all pointing to your site, still shouldn’t generate that sort of traffic. The effects are severe, but nothing like what CMDLN spoke of.

    The onus should be on the DNS providers to filter this sort of ridiculous traffic storm. Logging only tells you where the traffic came from; it won’t get money back. It would probably be a good idea to select a DNS provider that will agree to filter suspicious traffic for you or not hold you liable for a DOS attack.

    Good article, CMDLN, I enjoyed it.

  • Matt,
    Thanks for the comment. I do believe thats one of the best comments I have had yet. I do enjoy it when someone else validates one of my thoughts, but even more so when someone actually thinks and takes time to write a sensible reply.

    Hope you stick around, i pop by standalone-sysadmin from time to time good work, keep it up. There definitely are not enough admins sharing info!

  • At least one poster has said that the provider ought to be compensated for the bandwidth consumed in answering sudden increases in bogus requests. This is a real stretch because the bandwidth is on monthly commits, *and* dns takes *very* little bandwidth.

    While the overage charges are quite reasonable at dnsmadeeasy, the same is not true at ultradns/neustar. For the latter, overage rates could be termed punitive. So, the customer is stuck between buying a much larger subscription than normally required, or stand naked in the face of sudden surges.

    For the particularly evil minded, it might even seem possible that a commissioned sales rep on his own, or an organised effort, might be behind such scenarios. Just imagine being able to bump current earnings as well as simultaneously driving the customer towards a bigger monthly commit through the simple exercise of a few cpu cycles.

    As for the idea that a 1000 identical requests could come from a burst behind a firewall, that is just ludicrous. Client machines are pointed at dns caches. Often the one on the firewall. Well, that dns cache has it cached and does not need to get the record 999 extra times.

    In the end, throttling is good business. It is one element in evading dns ddos. This means that *other* customers on the dns servers are affected less. This in turn means that they *stay* customers after the ddos has subsided.

    Throttling cannot always be implemented without risk, but it has attractions to both the provider and customer. Never forget that the customer is not the only customer resident on the system. Collateral damage can be very expensive.

    edgedirector.com charges strictly on a pay for what you use basis at a single rate. And, query credits never expire, they just carry forward forever. You bought it, it’s yours. Not like prepaid cell phones and vanishing minutes.

  • Benjamin Krueger Ubuntu Firefox 3.0.3 wrote:

    Nick, there is a tricky problem with DNS throttling.

    Once you throttle your service, you open up another DoS attack. The UDP protocol is trivial to spoof. If I know I can get your service to blackhole new requests by submitting 10 requests per second, I can simply rewrite the source address on my request packets and cause anyone on the internet to be unable to reach your DNS servers.

    How much would it cost Amazon in lost productivity if their corporate headquarters could no longer lookup up production amazon.com nameserver entries? It would be easy to find out if the DNS hosting provider offered such a blackhole service.

  • Good point Ben. I hadn’t thought of that. That would still be a dos with a rather limited scope. You would be targeting specific people to not be able to lookup a specific domain.
    Also it would be my guess (may be not true) that Amazon has the cash to pay for the dns query overage in the first place. Not everyone that uses third party dns is Amazon. And while I had not thought of the case you bring up I was mostly appalled at the lack of choice. Without the option to throttle on a pay/query basis it seems too open for abuse. At least if you have the option for throttling you can choose how you pay for a query dos. Either pay for it by having some people not be able to access your site or pay for it when you get a huge bill in the mail.

  • I recently became the victim of this kind of attack. I use DNSMadeEasy vanity nameservers so the attacker thought they were only attacking my network. They gave up after less than an hour but that still led to over 2 million queries.

Leave a Reply

Your email is never shared.Required fields are marked *

To submit your comment, click the image below where it asks you to...
Clickcha - The One-Click Captcha