Block Ads & Malvertising on pfSense Using pfBlockerNG (DNSBL) – Old

Block Ads & Malvertising on pfSense Using pfBlockerNG (DNSBL) – Old

This walkthrough uses the DNSBL portion of pfBlockerNG to remove ads/advertising and more importantly, malvertising. It essentially creates a functionality similar to the pi-Hole project except it doesn’t require a separate piece of hardware. Instead, you just use your pfSense (pfBlockerNG)!

I love pfSense and if I could only install one package to enhance its capabilities, it would undoubtedly be pfBlockerNG. pfBlockerNG is a pfSense package maintained by @BBcan177 (on Twitter). It’s worth mentioning that BBCan177 has a Patreon campaign where you can easily donate a few bucks to ensure he continues maintaining and adding to the package.

pfBlockerNG adds all kinds of security enhancements that I’ve discussed previously such as blocking known bad IP addresses with blocklists (link below). If you don’t already have the blocklist functionality in place on your pfSense, I would strongly recommend adding it after you’re done with this walk-through.

Using pfBlockerNG (And Block Lists) On pfSense

Changelog
4Jan2018 – Originally posted
17Jan2018 – Added whitelist recommendations
25Jan2018 – Reworded ‘DNSBL firewall rule’ section
30Jan2018 – Added TLD blacklisting; Added warning about large lists and related memory issues (with unbound)
15Feb2018 – Added Spamhaus most abused TLDs info
3June2018 – Added link to new version of this walkthrough for the new version of pfBlockerNG
4June2018 – Added .cm to TLD block recommendations as well as DNS blocking section

<< Looking for a new version of this pfBlockerNG DNSBL guide? >>
Block Ads & Malvertising on pfSense Using pfBlockerNG (DNSBL)

Why remove advertising?

Advertising is great because it pays content creators for their work. After all, even this site utilizes Google Ads, albeit very very lightly. So why would I create a write-up on blocking ads? Because advertisements are known to carry malicious payloads and it’s impossible to distinguish between the two. To that end, I’ll happily sacrifice some money for the sake of improving security. 😉

Installation/Configuration

At this point, I’m assuming you have already installed the package. If not, go to System -> Package Manager and search for pfBlockerNG. The install should only take a minute or so depending on your internet connection and firewall. After installing the package, you will need to enable it from the main page (Firewall -> pfBlockerNG). The main change on this page is to click ‘Enable pfBlockerNG.’ You may note that I have other items checked as well and those are related to the blocklists/IPv4 configuration so they are not necessary at this time. Don’t forget to click ‘Save.’

Next, go to the DNSBL tab and it will take you to the default DNSBL landing page. Place a checkmark in ‘Enable DNSBL.’ If you only have one internal interface (such as LAN), then you shouldn’t need to do anything else. If you have multiple internal interfaces and you would like to protect them with DNSBL, then you will need to pay attention to the ‘DNSBL Firewall Rule’ section below. First, place a checkmark in the ‘DNSBL Firewall Rule’ box (red square below). Then, select the various interfaces (to the right) by holding down the ‘Ctrl’ key and left-clicking. Don’t forget to hit ‘Save’ at the bottom.

If your pfSense has plenty of memory, another really amazing feature to consider is TLD (below the DNSBL option in the picture below). This option is required for the TLD blacklists discussed later in the walkthrough. What does the TLD feature provide? Normally, DNSBL (and other DNS blackhole software) block the domains specified in the feeds and that’s that. What TLD does differently is it will block the domain specified in addition to all of a domain’s subdomains. As a result, a bad guy can’t circumvent the blacklist by creating a random subdomain name such as abcd1234.linuxincluded.com (if linuxincluded.com was in a DNSBL feed). That’s really powerful and as far as I know, it is one of the few DNS blackholing software that does it. If you’re unsure on your memory, this might be a feature to come back to after you get your feeds and everything else configured. Nonetheless, don’t sleep on this extremely powerful feature because TLD can definitely add several layers of protection.

pfBlockerNG DNSBL enable

DNSBL Feeds

Now, head over to the ‘DNSBL Feeds’ tab and click ‘Add.’ Once there, make DNSBL feed page resemble the one below. Below the image, I’ve provided the text below so you can easily copy/paste it into the page. To add more lines, click the ‘Add’ in the red box below. Once again, don’t forget to hit ‘Save’ at the bottom.

pfblockerng dnsbl feeds

SWC
http://someonewhocares.org/hosts/hosts
hpHosts
https://hosts-file.net/ad_servers.txt
Quidsup
https://raw.githubusercontent.com/quidsup/notrack/master/trackers.txt
Adaway 
https://adaway.org/hosts.txt
Cameleon
http://sysctl.org/cameleon/hosts

Assuming everything went as planned, when you clicked ‘Save’ you are taken back to the DNSBL feed list and it will look like the one below.

pfblockerng dnsbl feed list

Other/Malicious DNSBL Feeds

These are some other DNSBL feeds that I found to be very useful. To add them, you can either ‘edit’ the previously created DNSBL feed above by clicking the pencil next to the ‘Misc’ line item or simply add another. If you add a separate one, make sure you follow the settings above, although you can call the “DNS Group Name” malicious, largefeeds, pihole, or whatever else you like.

AbuseDOMBL
https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt

ISClow
https://isc.sans.edu/feeds/suspiciousdomains_Low.txt

Immortal
https://mirror1.malwaredomains.com/files/immortal_domains.txt

The feeds below are large, but they are very good feeds. If you using a system with limited resources (mainly RAM), then these might not be for you. When in doubt, add the feeds slowly and keep an eye on memory, CPU, etc. 

BBCDGAAgr
https://osint.bambenekconsulting.com/feeds/dga-feed.gz

hpHostsFSA
https://hosts-file.net/fsa.txt

Sites silently autofilling and extracting email addresses and other information for tracking. It’s based on the work from the Princeton folks found here.

Princeton
https://gist.githubusercontent.com/BBcan177/b6df57cef74e28d90acf1eec93d62d3b/raw/f0996cf5248657ada2adb396f3636be8716b99eb/MS-4

A list meant to prevent browser mining.

CoinlistBrowser
https://raw.githubusercontent.com/ZeroDot1/CoinBlockerLists/master/hosts_browser

These lists (in conjunction with two above) are what is used by default with the pi-hole project if you are trying to mimic it.

StevenBlack
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

MalwareDomains
https://mirror1.malwaredomains.com/files/justdomains

Zeustracker
https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist

DisconnectTracking
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt

DisconnectAds
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt

DNSBL Easylist

Next, go to the DNSBL Easylist tab. These are additional feeds that are simply a little easier to add. Make your screen look like the one below and then click ‘Save’ at the bottom. Note: To select all of the EASYLIST categories you need to hold down the “Ctrl” key while you left-click on each of them. It’s also worth mentioning they are privacy related selections in the EASYLIST. I found these to be mostly unusable because they broke several things (mainly related to Amazon), but your mileage may vary.

pfblockerng dnsbl easylist

Now, go over to the Update tab within pfBlockerNG. Heed the warning in the first red box and make sure you are not going to run the updates near the time your cron job would automatically run. If the countdown timer is less than 3 minutes, I would not recommend running it and instead just wait for the system to run it automatically. Assuming you are good on the time, go ahead and click the ‘Run’ button. You will see progress updates in the gray window below including the number of domains downloaded by each list. Also note that pfBlockerNG is smart enough to check for and eliminate duplicate DNS (# Dups) entries between the lists.

pfblockerng dnsbl download and update

[ SWC ] Downloading update .. 200 OK.
 Whitelist: localhost.localdomain|
 ----------------------------------------------------------------------
 Orig. Unique # Dups # White # Alexa Final 
 ----------------------------------------------------------------------
 13170 13165 310 1 0 12854 
 ----------------------------------------------------------------------

[ hpHosts ] Downloading update [ 01/04/18 15:58:40 ] .. 200 OK.
 ----------------------------------------------------------------------
 Orig. Unique # Dups # White # Alexa Final 
 ----------------------------------------------------------------------
 47811 47808 3658 0 0 44150 
 ----------------------------------------------------------------------

[ Quidsup ] Downloading update [ 01/04/18 15:58:44 ] .. 200 OK
 Remote timestamp missing .
 ----------------------------------------------------------------------
 Orig. Unique # Dups # White # Alexa Final 
 ----------------------------------------------------------------------
 12570 12570 714 0 0 11856 
 ----------------------------------------------------------------------

[ Adaway ] Downloading update [ 01/04/18 15:58:45 ] .. 200 OK.
 ----------------------------------------------------------------------
 Orig. Unique # Dups # White # Alexa Final 
 ----------------------------------------------------------------------
 409 409 280 0 0 129 
 ----------------------------------------------------------------------

===[ DNSBL Domain/IP Counts ] ===================================
78530 total
 44150 /var/db/pfblockerng/dnsbl/hpHosts.txt
 12854 /var/db/pfblockerng/dnsbl/SWC.txt
 11856 /var/db/pfblockerng/dnsbl/Quidsup.txt
 9541 /var/db/pfblockerng/dnsbl/Easy.txt
 129 /var/db/pfblockerng/dnsbl/Adaway.txt
===============================================================

Testing By Browsing

So what does the finished product look like? On YouTube, you’ll see that gray boxes like the one shown below where an ad normally would have been. A browser add-on like uBlock Origin (discussed below) further cleans this up by removing the gray box entirely and it also provides some secondary protections.

pfblockerng Youtube ads gone

If you visit Yahoo.com (why? seriously, find a new news site), our pfBlockerNG configuration eliminates the wasteland of ads that you normally see as well (red box below).

pfblockerng yahoo ads gone

How it works – testing from the command line

So you see the end result when browsing, but what’s really going on? How the DNSBL portion of pfBlockerNG works is most easily seen via a command line. Normally, you would ping 302br.net and get back their actual IP address. However, with pfBlockerNG properly setup you will instead see a reply of 10.10.10.1, which is the default virtual IP address DNSBL creates. Basically, the ad/malvertising domain name is blackholed instead of displaying (or resolving). Feel free to test this against any domain in any one of the lists that you added. If you followed all of my examples above for both ads and malicious sites, you will likely have a DNSBL list that is well into the hundreds of thousands if not millions.

Integral Ad Science

# ping 302br.net
PING 302br.net (10.10.10.1) 56(84) bytes of data.
64 bytes from 10.10.10.1 (10.10.10.1): icmp_seq=1 ttl=64 time=0.684 ms

Yahoo – analytics.yahoo.com

# ping analytics.yahoo.com 
PING analytics.yahoo.com (10.10.10.1) 56(84) bytes of data.
64 bytes from 10.10.10.1 (10.10.10.1): icmp_seq=1 ttl=64 time=1.18 ms

Troubleshooting/Whitelisting

What happens if/when a website is inadvertantly blocked? Afterall, it is bound to happen. You can either remove the offending list entirely or more preferably, you can just whitelist the domain. The absolute easiest way to do this is by going back to the main DNSBL tab and clicking the ‘+’ to reveal a textbox for input. Simply type each domain in on a separate line and then click ‘Save’ when you are done. If you want the changes to occur sooner rather than later, go back to the ‘Update’ tab and click ‘Run.’ If you don’t want to do the trial and error on your own, I have provided some whitelist recommendations below.

pfblockerng DNSBL whitelist options

It’s also worth mentioning that if a system already resolved the domain name and it is now resolving to 10.10.10.1, then you may need to clear your local DNS cache, your browser cache, or both. To clear your machine cache, from a command line on Windows, type in ‘ipconfig /flushdns’ and that should take care of it. You can do the same on a Linux system, although the commands can vary from one installation to the next. More often than not, simply restarting your network interface will work. Thus, on most distributions, ‘service networking restart’ or ‘systemctl restart network’ should take care of it for you. Each browser has a slightly different way to clear the cache, however, all of them allow you to pull a new version of the website if you hold down “Shift” while clicking on the refresh/reload button.

If ads are not getting blocked and the ping commands above don’t return the virtual IP address, it’s also possible your local machine is not using pfSense for its DNS settings. If you are using Windows, check your network settings and make sure it is set to your pfSense IP address. On Linux/*nix, check your /etc/resolv.conf or even Network Manager (if using a GUI). If you are not using pfSense for your DHCP server, you may need to do some digging. Although somewhat uncommon, keep in mind that some anti-virus packages can mess with DNS configuration settings too.

Whitelist Recommendations

These are a few domains I’ve seen cause issues if they end up on the various DNSBLs. You can easily copy and paste them into the “custom list” as described above. If you have no plans to use some of them (based off their name alone), feel free to omit them from your list. Do you have other recommendations beyond the ones I have listed? Let me know and I’ll add them!

.amazon-adsystem.com
.adsafeprotected.com
control.kochava.com
device-metrics-us-2.amazon.com
secure-gl.imrworldwide.com
.githubusercontent.com
.github.com
github.map.fastly.net 
.apple.com
.sourceforge.net
s3.amazonaws.com
s3-1.amazonaws.com 

TLD Blacklisting

TLD (top-level domain) blacklisting is another option in DNSBL. Don’t forget you need to ‘Enable’ the TLD option at the top of the DNSBL configuration page to use the features discussed here. While I don’t normally advocate static blacklisting because the bad guys will simply move around it, TLD blacklisting is a rare instance where you can eliminate some potential attack vectors although its usefulness depends entirely on your situation. TLDs are the characters after the last dot on a domain name, e.g. com, net, and biz are some common ones. The number of TLDs has skyrocketed and there were well over 1,500 in early 2017! Over time, some TLDs have become wastelands for nefarious activity such as command and control servers. If you no plans to connect with a particular TLD and it has shown to be less than reputable, i.e. most sane companies wouldn’t bother trying to use it, you can just go to the main DNSBL tab and block it outright using the section below.

DNSBL blacklist

Even Brian Krebs got in on talking about the how some TLDs are used extensively for typosquatting — Omitting the “o” in .com Could Be Costly. If you don’t want to read the full article, just understand that instead of typing in remax[dot]com, a user mistakenly types in remax[dot]cm and is directed to a malicious site. There are similar alternative .cm domains for ESPN, Hulu, iTunes, Aetna, AOL, Chase, Facebook, WalMart, etc. and over 1000 others. Needless to say, the .cm TLD is not good.

If you’re looking for a little more guidance of what is ‘bad’ then look no further than Spamhaus and the website link below. Spamhaus is constantly updating this list and related statistics so check it directly for the most up-to-date information. At the link below, you’ll also find a dropdown to show you the ‘badness’ of every TLD even beyond the top 10 list. At the very least, I would suggest adding the top 3 TLDs in the green box below along with the .cm TLD from the Krebs article. Adding the entire top 10 would likely not cause too many issues, although keep in mind that you will see false positives. For example, note that #8 in the list below is .biz, which is used by legitimate businesses. I’ve added a textual version of the TLD list below the image so you can easily copy/paste it into your firewall.

https://www.spamhaus.org/statistics/tlds/

Spamhaus most abused TLDs

cm
party
click
link
technology
gdn
study
men
biz
reise
stream

DNS blocking

DNS blacklists are great, but what if a new ransomware botnet pops up, a user gets infected fairly early in the campaign, and it starts calling previously unknown domains? If your DNSBL feeds are set to update every 4 hours and it takes time for them to get included on that list to begin with, it might take awhile before your DNS catches and blocks it. We need something more real-time… To provide another layer of protection, I would also recommend using Quad9 as your primary DNS on pfSense. I wrote up an article some time ago about how to do just that.

Configuring Quad9 on pfSense

Browser side blocking – Ublock Origin

I constantly preach defense-in-depth and this is no different. You could have every malicious advertising domain on the planet included in your configuration, but a new one will inevitably pop-up 5 minutes from now. Aside from some other defenses, I would also strongly suggest using uBlock Origin on all of your browsers. uBlock Origin exists for Chrome, Firefox, etc. so there really isn’t a reason not to have it! While nothing is foolproof, it is another fantastic addition to your overall security.

uBlock Origin

Dallas Haselhorst has worked as an IT and information security consultant for over 20 years. During that time, he has owned his own businesses and worked with companies in numerous industries. Dallas holds several industry certifications and when not working or tinkering in tech, he may be found attempting to mold his daughters into card carrying nerds and organizing BSidesKC.

35 thoughts on “Block Ads & Malvertising on pfSense Using pfBlockerNG (DNSBL) – Old

  1. Nice guide. THe other thing I do for monitoring is clear the dnsbl.log file. Run an update, then download the log file. Pull the summary information and you can see what filters are doing the heavy lifting. The filters will change position between successive runs because another filter may capture a domain first on one run and be blocked on another. You can see which filters are not adding additional value.

    1. Glad you enjoy it and good tip! This is slightly easier in the upcoming release too because it provides stats on what is getting blocked too. I’ll write up a new guide once it is released.

  2. Enabling this blocks all amazon(with alexa enabled) and all apple sites. Only using EasyList. Also, I dont want to make it sound like your guide isn’t complete or lacking. You have a pretty good guide here! I just don’t understand why these other services always break when this service is enabled.

    1. Robert, thanks for the feedback. As I noted above, issues with Amazon is the main reason I avoid using the privacy lists. I just tested the setup and I was able to get Alexa to access Pandora with all of the lists mentioned above running (related DNS traffic below). If you’re seeing something different, please let me know so I can get it changed for other readers. Thanks!

      22:15:42.978282 IP myip.20990 > 198.51.45.5.53: UDP, length 60
      22:15:43.004471 IP 198.51.45.5.53 > myip.20990: UDP, length 127
      22:15:43.004745 IP myip.24799 > 199.116.165.20.53: UDP, length 66
      22:15:43.073464 IP 199.116.165.20.53 > myip.24799: UDP, length 248

  3. I had a question about the DNSBL Firewall Rule section, In my setup I have multiple Interfaces on my LAN side. I basically separate user computer from VMs that run services. I’m a bit confused on this area. Am I supposed to select the interfaces I want to protect and then check the box? The Image highlights that check box, but shows it unchecked. Am I supposed to select interfaces only or select interfaces AND check that box? Sorry if this is a super basic question.

    1. Not a basic question at all and I applaud you for segmenting your internal network! Under the primary DNSBL tab, you will see options for “DNSBL Listening Interface” and “DNSBL Firewall Rule.” These are the options you need to focus on. Assuming your primary interface is LAN, you can leave the DSNBL listening interface dropdown alone. Now, put a checkbox in the DNSBL firewall rule option, select your internal interfaces (using ctrl + left clicking) and then hit ‘save’ at the bottom. Let me know if you need anything else!

  4. Thanks for taking the time to write this. It’s just the kind of hand-holding a rank amateur like me needed. I was going through the pfsense documentation and was getting kind of overwhelmed.

  5. Hello,

    Which “list action” do you choose here:

    DNSBL IP Firewall Rule Settings
    Configure settings for Firewall Rules when any DNSBL Feed contain IP Addresses

    Thanks,

    1. Steve, thanks for stopping by! The guide on blocklists (DNSBL IP) is here: https://www.linuxincluded.com/using-pfblockerng-on-pfsense/ The list action I would recommend is deny both. This will block and generate an alert whether the offending IP is coming in (WAN) or going out (LAN). Keep an eye out for outbound alerts as that means an internal machine is trying to communicate with a known bad IP. Maybe it is a false positive or maybe it is something more… Nonetheless, it’s worth investigating and a great start to threat hunting.

  6. Thanks Dallas, great article. There are a few tutorials on this subject, but this is the most clear and easy to follow

  7. After I follow your configuration in DNSBL Feeds I ecountered error this is the error

    ****The following input errors were detected:

    -Header field cannot contain special or international characters.
    *****

    1. Hey Reynan! I haven’t seen that error. Can you verify you are not using any special characters in the header/label field? If I remember correctly, I’ve used underscores in the past without issue, although I opted to use camel case (alphanumeric only) in the guide specifically to avoid all special characters.

      Update: I noticed the one exception to the statement above was my usage of an underscore in the “Coinlist_browser” header/label. I changed this to “CoinlistBrowser” in the guide because consistency is good. FWIW, I was able to reproduce the error you mentioned above by trying to use any other special character (beyond an underscore “_”) in the header/label field.

      1. I received the same error when I copied/pasted “BBCDGAAgr” into the Header/Label field. After deleting and retyping the same text everything worked ok.

        1. I searched for special characters on all of the field names including BBCDGAAgr and I couldn’t find anything. I’m suspecting an “extra” trailing space is the issue when copying/pasting the names since the field in question has no other special characters. Nonetheless, happy to hear everything is working for you now!

    1. Jerome, great question! I don’t use the setup you describe, however, I have looked at it before. I’m assuming you are referring to leaking as external WAN DNS traffic not over the VPN? Without seeing the exact setup, here are a couple of items to check out. First, have you verified the “DNS Server Override” is not checked in the general setup? Second, when you connect via your OpenVPN client, PIA sets the DNS on your TAP adapter (like below). To have the firewall perform in a similar fashion over PIA, you will likely need another outbound NAT, e.g. one for the virtual IP to LAN.

      Another alternative to avoid DNS data leakage would be to configure DNS over TLS. It doesn’t appear PIA supports DNS over TLS, however, services such as Quad9 and CloudFlare both support it. I’ve written about configuring the latter on pfSense if you are interested — https://www.linuxincluded.com/configuring-quad9-on-pfsense/ . If I ever get around to testing this out myself, I’ll let you know as well.

      Ethernet adapter Ethernet 2:
      Connection-specific DNS Suffix . :
      Description . . . . . . . . . . . : TAP-Windows Adapter V9
      Physical Address. . . . . . . . . :
      DHCP Enabled. . . . . . . . . . . : Yes
      Autoconfiguration Enabled . . . . : Yes
      IPv4 Address. . . . . . . . . . . : X.X.X.X(Preferred)
      Subnet Mask . . . . . . . . . . . : 255.255.255.252
      Lease Obtained. . . . . . . . . . : Monday, May 7, 2018 3:30:29 PM
      Lease Expires . . . . . . . . . . : Tuesday, May 7, 2019 3:30:29 PM
      Default Gateway . . . . . . . . . :
      DHCP Server . . . . . . . . . . . : X.X.X.X
      DNS Servers . . . . . . . . . . . : 209.222.18.222
      209.222.18.218

  8. 1) Thnx for the hints. DNS over TLS is of course the preferred option, but I finally solved the DNS leak with the PIA VPN basic settings by changing Services/DNS resolver/General settings/Outgoing Network Interfaces to LAN only (instead of WAN).

    2) uBlock, your comments on uBlock inspired me. I simply included this https://raw.githubusercontent.com/IDKwhattoputhere/uBlock-Filters-Plus/master/uBlock-Filters-Plus.txt as an extra DNSBL feed. The result was amazing. I got rid of another bunch of add’s.

    I used the pi-hole test page: https://pi-hole.net/pages-to-test-ad-blocking-performance/

    1. Jerome, thanks for the heads up!
      1) It’s interesting your WAN vs. LAN worked. Do you have a firewall rule that allows DNS on the WAN side?
      2) That uBlock Origin add-on looks interesting. I tend to let it operate in its default state and leave a majority of blocking to pfSense/pfBlockerNG. Letting pfBlockerNG do its thing is easier with the next version as it makes adding blocklists trivial. On my test system, I literally have 60+ DNSBLs. I received notification from the author that it was sent to pfSense development so it probably isn’t too far off from final release.
      That website looks like a decent way to test ad blocking. I usually visit the news wasteland aka Yahoo as well as a few others. FWIW, a malvertising ad on Yahoo actually serves as the featured image for this post. The backstory to it is that when I was writing up this how-to, I spun up a brand new VM, visited Yahoo (and only Yahoo) with a up-to-date Chrome browser and no ad blockers (uBlock Origin or pfBlockerNG)… About 30 seconds after landing on Yahoo, I received that pop-up. We forgot how difficult/impossible the web is to use for the average user who has no ad blocking.

  9. Great guide Dallas! I was looking for a good guide when I found this. Was able to set it up and get in running in less than 15 minutes. Thanks for that.

  10. Excellent guide dallas, thanks for that. I too had a problem with dns leak using expressvpn. Thanks jerome for the fix with changing it to lan. Worked for me too.

    1. Excellent! I haven’t got around to testing Jerome’s solution, but I’m happy to hear it worked for you too. I’ll add something into the next version of this documentation when the new version of pfBlockerNG moves from devel to stable. Thanks for the feedback!

Leave a Reply

Your email address will not be published.