Configuring Quad9 on pfSense

Configuring Quad9 on pfSense

Quad9 is a DNS platform that adds several layers of security. It does this via standard DNS queries/responses.Basically, if a machine on your network queries a known bad hostname, the Quad9 DNS server responds by stating that domain does not exist (NX DOMAIN or non-existent domain). Quad9 also allows you to use DNS over TLS. If you would like a bit more info on Quad9 including some speed benchmarks against other DNS services, I would suggest an earlier article, Quad9 – First Thoughts & Benchmarks.

This post is all about configuring, testing, and troubleshooting Quad9 on pfSense, although many of the same rules apply to nearly any firewall on the market. The DNS resolver on most pfSense distributions is unbound so this documentation was written as such.

Changelog
29Nov2017 – Originally posted
26Feb2018 – Added unbound option “log-replies: yes” so source IP is recorded with NXDOMAIN response
10Mar2018 – Changed IPv6 secondary address based on feedback
6Apr2018 – Added DNS over TLS section

pfSense Configuration

First things first, after logging into your firewall, go to System -> General Setup so you can change your primary and secondary DNS servers to those of Quad9. While you are there, make sure the “DNS Server Override” and “Disable DNS Forwarder” options are not checked (as shown below). Don’t forget to click ‘Save’ at the bottom.

IPv4
Primary DNS: 9.9.9.9

Secondary DNS: 149.112.112.112
IPv6
Primary DNS: 2620:fe::fe
Secondary DNS: 2620:fe::9 <- Quad9 states this is not operational at the moment
The secondary DNS for IPv6 is not permanent. I’ll try to update it when the change occurs, but keep an eye on the Quad 9 FAQ for the most up-to-date info. 

Warning! Do not get cute and add in a tertiary DNS *or* think you know better by adding in OpenDNS or Google as a secondary DNS for improved redundancy. If you do this, you will get unexpected results. The DNS servers are *not* queried in order as you might expect. Instead, the servers are either asked simultaneously (dnsmasq) or they flip flop who is the “primary” DNS frequently. In my testing, the Google DNS was just a titch faster so it stayed primary more often than not. In a handful of tests, the 1st query was sent and it didn’t come back fast enough so Google DNS was queried second. Surprisingly, Google DNS still answered first in some instances despite its later start. As a result, the IP address resolved as it normally would and the second/later response of NX Domain was discarded. At any rate, any DNS configuration other than what I have stated above breaks the blocking features of Quad9, which is the main reason for using it IMO.

Change pfSense DNS servers to Quad9

Next, go to the Services -> DNS Resolver and put a checkmark in the “DNS Query Forwarding” if it isn’t there already. You can leave the DNSSEC box checked too. Click ‘Save’ and then click ‘Apply Changes’ at the top.

pfSense DNS resolver

Before moving on, I would strongly suggest following the “Log Configuration” section toward the bottom of this post. While it is not 100% necessary, it will make your life easier in the future and it is highly recommended!

Testing/Troubleshooting

Below are some various ways to test your configuration whether in Linux/FreeBSD, Windows, or from the browser using the isitblocked.org domain.

Linux command line – Not working

# dig +short isitblocked.org
74.208.236.124

Linux command line –Working

# dig +short isitblocked.org

Nothing is returned above.

Windows command line – Not working

C:\Users\User>nslookup isitblocked.org
Non-authoritative answer:
Name:    isitblocked.org
Addresses:  2607:f1c0:100f:f000::2d1
          74.208.236.124

Windows command line – Working

C:\Users\User>nslookup isitblocked.org
*** can't find isitblocked.org: Server failed

Browser

Assuming your configuration is correct, from a browser the isitblocked.org domain should appear as though it is down (below). If you see anything else or your browser gets re-directed to http://www.isitblocked.org/default, then something isn’t working right.

NX DOmain firefox

More Troubleshooting – Present & Future

Either your configuration isn’t working or you realized that at some point, you’re going to need to troubleshoot a domain getting blocked and you wanted to read ahead. 😉 One way to test your new DNS configuration is via the web GUI by going to Diagnostics -> DNS Lookup. Once again, type in isitblocked.org and see the response. If you see a “could not be resolved” as shown in yellow above, congrats! The only issue? That doesn’t necessarily mean everything is working. The DNS lookup via the web GUI performs a sequential lookup of DNS servers. Remember the red warning above? Nonetheless, this can still be a useful step in your troubleshooting endeavors.

pfSense DNS lookup via web gui

For those interested, this is what the “race” looks like during a packet capture despite 9.9.9.9 being “above” 8.8.8.8 in the DNS configuration screen and getting queried first.

IP MYEXTIP.53125 > 9.9.9.9.53 <-- FIRST QUERY (TO QUAD9)
IP MYEXTIP.54417 > 8.8.8.8.53 <-- SECOND QUERY (TO GOOGLE)
IP 8.8.8.8.53 > MYEXTIP.54417 <-- FIRST RESPONSE (FROM GOOGLE)
IP 9.9.9.9.53 > MYEXTIP.53125 <-- SECOND RESPONSE (FROM QUAD9)

Log Config – NXDOMAIN

What’s the best way to troubleshoot? Get familiar with your logs. Better yet, configure them properly when you first set this up. Go to Services -> DNS Resolver -> Advanced Settings and switch the default log level of 1 to 2 (or higher). Don’t forget ‘Save’ and ‘Apply Changes’ once you are done.

pfSense log level on DNS resolver

Now, you can go to your system logs (Status -> System Logs -> System -> DNS Resolver) and see every DNS query/response. Keep in mind the logs will also show which domains are returning as NXDOMAIN.

pfSense DNS Resolver NXDOMAIN

Log Config – Add requesting/source IP to NXDOMAIN response

Because of how Quad9 responds for malicious domain queries, you can see first-hand if any devices on your network are trying to contact known bad guys on the internet. By default though, the originating/source IP address is not recorded with the DNS NXDOMAIN response. To correct this, add “log-replies: yes” to the “Custom options” section of the general settings page of your DNS Resolver configuration (Services -> DNS Resolver) as shown in the highlighted section below. If you have other custom options there (such as one added by pfBlockerNG), simply add the log-replies option on a new line. Don’t forget to hit Save then Apply after you’ve finished adding it. It’s also worth mentioning the unbound documentation warns when using either the “log-queries” or “log-replies” options. From the unbound documentation, “Note that it takes time to print these lines which makes the server (significantly) slower.” I enabled “log-replies” and I didn’t see a discernible increase in disk I/O, CPU, memory, etc. but your mileage may vary.

log-replies: yes

Unbound Log Replies Option pfSense

Here are what the logs (Status -> System Logs -> System -> DNS Resolver) look like when querying a random hostname. These are the same logs in your SIEM if you forward your pfSense logs as well.

Feb 26 14:53:08 unbound 91442:0 info: control cmd: stats_noreset
Feb 26 14:53:08 unbound 91442:1 info: [my IP address] abbbbbbbbbaaaa22222233333333322244422.com. A IN NXDOMAIN 0.042276 0 132
Feb 26 14:53:08 unbound 91442:1 info: validate(nxdomain): sec_status_insecure
Feb 26 14:53:08 unbound 91442:1 info: query response was NXDOMAIN ANSWER


Unbound log-replies nxdomain

Add Security To DNS with DNS over TLS

Another security enhancement Quad9 offers that many other DNS providers do not is DNS over TLS. Using this option forces your pfSense firewall (unbound resolver) to encrypt the DNS transaction as it traverses the internet, i.e. a man-in-the-middle on the internet (or a nosy upstream network provider) can’t see which hostnames you are querying and as important, no one can modify the DNS responses.

If you’ve followed the logging directions above, you’ll see the example below has the “log-replies” setting. If there are other settings in the custom options, you can safely place these additional options below them.

DNS over TLS pfSense

If you want to copy/paste everything including the log-replies settings, here they are! Make sure you save and apply your settings.

server:
log-replies: yes
forward-zone:
name: "."
forward-ssl-upstream: yes
forward-addr: 9.9.9.9@853
forward-addr: 149.112.112.112@853

You can test this by performing a packet capture on your firewall. Go to Diagnostics -> Packet Capture to get started. Just make sure your interface is set to WAN and add 9.9.9.9 as your host address before you start your capture or you’ll get tons of other data. As you can see in the simple connection data below, your queries/responses are now over TCP port 853 (DNS over TLS) instead of the standard DNS query/response (UDP port 53).

23:01:41.410868 IP myip.46451 > 9.9.9.9.853: tcp 0
23:01:41.439209 IP 9.9.9.9.853 > myip.46451: tcp 0
23:01:41.439266 IP myip.46451 > 9.9.9.9.853: tcp 0
23:01:41.439420 IP myip.46451 > 9.9.9.9.853: tcp 307
23:01:41.467633 IP 9.9.9.9.853 > myip.46451: tcp 1448
23:01:41.467710 IP myip.46451 > 9.9.9.9.853: tcp 0
23:01:41.467715 IP 9.9.9.9.853 > myip.46451: tcp 1448
23:01:41.467732 IP myip.46451 > 9.9.9.9.853: tcp 0
23:01:41.467736 IP 9.9.9.9.853 > myip.46451: tcp 42
23:01:41.467784 IP myip.46451 > 9.9.9.9.853: tcp 0

Taking a peek at the saved packet capture in Wireshark, you’ll see more connection data including the TLS handshake. What you won’t see are hostnames, responses, or any other standard query/response data you would find in a normal, clear-text DNS transaction!

Wireshark - Quad9 DNS over TLS transaction

Dallas Haselhorst has worked as an IT and information security consultant for over 20 years. During that time, he has owned his own businesses and worked with companies in numerous industries. Dallas holds several industry certifications and when not working or tinkering in tech, he may be found attempting to mold his daughters into card carrying nerds and organizing BSidesKC.

8 thoughts on “Configuring Quad9 on pfSense

  1. If I get an NXDOMAIN answer, is there any way to see which of my home IPs generated it? Currently the logs dont seem to list the originating IP address ?

    1. Great question! In the past I tried using unbound options to do this, e.g. “log-replies: yes” and “log-queries: yes” since those are native options. Unfortunately, I could never get them to work whether I used the “custom options” field (Services -> DNS Resolver) or added it to existing config files. My workaround was simply creating a firewall rule for port 53 and enabling logging on that rule. This does require some event correlation so it’s not ideal. Now that unbound has been around a bit longer on pfSense, I’ll dig around a bit more.
      — UPDATE —
      After thinking about this a bit I figured it out. In the “custom options” you also need to include “server:” before it. Thus, adding “server:log-replies: yes” will provide the desired outcome — the source IP address and the NXDOMAIN response on a single line (example below). I’ll update the documentation above since I believe this is an extremely helpful suggestion. Also, heed the warning about printing this information makes unbound “significantly slower” according to their own documentation (link below). Please let me know if you have any issues!
      Feb 26 13:01:30 unbound 90310:1 info: [my internal IP address] aaaaaaaaaaa222233333333333312222222222222222.com. A IN NXDOMAIN 0.434214 0 132
      https://www.unbound.net/documentation/unbound.conf.html
      — UPDATE —
      I tried this again without the “server:” directive and it works now as well. Thus, “log-replies: yes” or “server:log-replies: yes” should work in the “Custom options” field. Make sure it is on a new line and click save and apply.

  2. How can you get DNS over TLS to work with pfblockerng dnsbl filtering? i want to filter using this, but then any queries my resolver makes that is legit, i would like it to use the dns over tls.

    1. Micheal, DNSBL and the DNS over TLS with perfectly together. In fact, this is exactly how I have it configured on all my installs. Just make sure you leave the DNSBL-related line in the unbound configuration. If you get any errors during the initial setup, cut the DNSBL line out of the config, save it, and then re-add it. Feel free to holler if you have any questions!

      1. Ok i was wondering bc in your help you said to turn on dns query forward, but in the dnsbl help document it says to not have it on. So which way am i supposed to have it for both to work properly?

        1. I tested both checked/unchecked on the “DNS Query Forwarding” option and Quad9 successfully blocked the test site, isitblocked.org, in both instances. That said, all my firewalls have “Enable Forwarding Mode” checked. I based this off the documentation below. If you find anything different in your setup, please let me know!

          Enable Forwarding Mode: Controls whether Unbound will query root servers directly (unchecked, disabled) or if queries will be forwarded to the upstream DNS servers defined under System > General or those obtained by DHCP/PPPoE/etc (checked, enabled). Forwarding mode may be enabled if the upstream DNS servers are trusted and also provide DNSSEC support.

Leave a Reply

Your email address will not be published.