If you have a sizable quantity of mail delivery, like greater than 20k messages per day, you are advised to run your own caching DNS server as it benefits your Spamassassin deployment in both speed and reliability. This article describes why it is important to operate your own local caching DNS resolver, and how to enable it on your server.
Spamassassin-3.3.2 has been released! [READ MORE]
Why is it important to run your own DNS server?
Spamassassin makes many DNS queries during scanning. It tests various IP-based blacklists, URI-based blacklists and other network queryable rules. If you run these commands you will see how long it takes individual DNS queries to complete.
wget http://people.apache.org/~wtogami/sample-spam.eml
cat sample-spam.eml | spamassassin -D 2>&1 \
| grep ‘async: timing’ | sed ‘s/^.*dbg: async: //’
You should see output similar to this.
timing: 0.012 . dns:A:45.135.176.118.bl.spameatingmonkey.net.
timing: 0.013 . dns:A:45.135.176.118.iadb.isipp.com.
timing: 0.015 . dns:A:technodiva.com.fulldom.rfc-ignorant.org.
timing: 0.015 . dns:A:45.135.176.118.ix.dnsbl.manitu.net.
timing: 0.016 . dns:TXT:45.135.176.118.sa-accredit.habeas.com.
timing: 0.018 . dns:A:45.135.176.118.bb.barracudacentral.org.
timing: 0.019 . dns:A:45.135.176.118.bl.mailspike.net.
timing: 0.020 . dns:A:45.135.176.118.list.dnswl.org.
timing: 0.021 . dns:A:45.135.176.118.psbl.surriel.com.
timing: 0.022 . dns:A:45.135.176.118.bl.score.senderscore.com.
timing: 0.023 . dns:MX:technodiva.com
timing: 0.023 . dns:A:technodiva.com
timing: 0.023 . dns:A:technodiva.com.rhsbl.ahbl.org.
timing: 0.023 . dns:A:45.135.176.118.zen.spamhaus.org.
timing: 0.024 . dns:A:45.135.176.118.dnsbl.sorbs.net.
timing: 0.025 . dns:A:45.135.176.118.bl.tiopan.com.
timing: 0.026 . dns:A:45.135.176.118.dnsbl-1.uceprotect.net.
timing: 0.026 . dns:A:45.135.176.118.combined.njabl.org.
timing: 0.195 . dns:TXT:45.135.176.118.bl.spamcop.net.
The second column shows how long in seconds it took for that particular query to succeed. This particular example scan was very quick as the DNS cache was already primed with these particular queries. Quite often however, you may find that your Spamassassin DNS queries take much longer (several seconds), intermittently fail, or time out on a permanent basis. Spamassassin relies *heavily* on reliable and fast DNS lookups, so it is imperative to proactively fix and prevent DNS issues.
It is often the case that the DNS servers provided by your home ISP or server host will not behave properly. Their servers might be overloaded, misconfigured to inappropriately cache entries, or intentionally exhibit certain non-standard misbehavior. Fortunately, nothing stops you from running your own caching DNS resolver server. Doing so completely avoids problems that can be caused by accidental or intentional misbehaviors of someone else’s DNS server.
A more important reason to use your own DNS server rather than rely on your server provider is due to the usage limitations of free network rules. Say your mail server is one of thousands of independent mail servers hosted at a data center. Your Spamassassin DNS queries may be aggregated with other unrelated mail servers and appear to all be coming from the same IP address. This can be bad, because some of those network rule providers have limits of like 100k DNS queries per day for free users. This can be seen as abuse of their terms of service, and such IP addresses can be blocked or blackholed. Such a block can cause significant problems for your Spamassassin server and mail delivery pipeline as it degrades your spam filter and may even cause significant delivery delays.
How to run your own local caching DNS resolver server
There are many different options for caching DNS servers. The daemon that I have used is pdns-recursor, which claims to be fast and efficient. Others have suggested trying unbound, but I haven’t tried it myself yet.
If you are running Fedora, or RHEL/CentOS/Scientific Linux with EPEL, you can use these commands to enable pdns-recursor.
yum install pdns-recursorchkconfig pdns-recursor on
service pdns-recursor start
Then edit your /etc/resolv.conf to make 127.0.0.1 your primary DNS server. If your distribution has a service like NetworkManager configuring your network interfaces, you may need to edit the configuration there to force it to 127.0.0.1 instead of whatever DNS server it is given via DHCP.
[root@xenguest ~]# rec_control get-all
cache-entries 994382
cache-hits 290981
cache-misses 13142625
packetcache-entries 299330
packetcache-hits 18023035
packetcache-misses 13433477
After you have run your Spamassassin server utilizing your local pdns-recursor for a while, you can examine its runtime statistics to get some useful statistics. The example above is a pdns-recursor instance running from May 7th to July 4th, 2011 on a small Spamassassin server. I have trimmed the output above to the important numbers demonstrating the cache efficiency. The third reason why it would be beneficial to run your own caching DNS server is because it substantially reduces the quantity of outbound DNS queries. This saves you bandwidth, time, and quite possibly will keep you below the usage limits of free network rules.
Memory Usage Considerations
There is a drawback to running pdns-recursor. The above pdns-recursor instance is using ~400MB of memory. If you cannot afford this kind of memory use, you can reduce the limits in options max-cache-entries and max-packetcache-entries in /etc/pdns-recursor/recursor.conf as documented upstream. You will need to find a balance between memory use and effective cache hit performance.
It is entirely possible that this described configuration of pdns-recursor could be improved, but I haven’t tested any of these theories. For example, DNS results of DNSBL and URIBL’s are very transient in nature with tiny TTL’s, so perhaps we could substantially reduce memory usage by forcing max-cache-ttl and max-negative-ttl to a much smaller duration. It also appears that the packetcache is far more effective than the cache at achieving hits, so we may be better off favoring the packetcache rather than the memory hogging and less effective cache.
Update July 5th, 2011: I actually tested this theory by reducing those two parameters to 3600 seconds. It substantially reduced the amount of memory used by pdns-recursor while not effecting the cache hit rate. This seems to be because pdns-recursor is keeping lots of old and useless entries in memory, while only more recent entries seem to matter. In any case, the packetcache seems to be most of the benefit, so it may help to increase the max-packetcache-entries parameter.
Update July 6th, 2011: I have discovered that pdns-recursor in EPEL5 is dangerously out of date. All versions prior to 3.3 has a subtly broken packetcache behavior that can degrade spamassassin’s filtering effectiveness. Unfortunately, EPEL5 cannot upgrade to a newer version of pdns-recursor because it relies on a newer version of Boost that cannot be upgraded. So EL5 users are advised to use a different DNS resolver.
Do you know if alternative caching DNS resolver daemons are more memory efficient than pdns-recursor? Please comment below!
Subscribe to the Spamassassin for Sysadmins Newsletter for occasional news important to your Spamassassin deployments. Also see our Ultimate Setup Guide for the latest tweaks to maximize your spam filtering effectiveness and safety.
