Kuuko / Build it yourself

Build it yourself · Part 1Checked: 30 October 2026

Chapter 6: Block lists that update themselves

Why

Your resolver blocks a tracker by answering "this name doesn't exist". Which names those are comes from block lists. Trackers, fake shops and phishing sites change every day, so a list only protects you if it's current. And a broken download must never leave you with an empty list, or a list that blocks everything.

Our choice, and the alternatives

We use the HaGeZi DNS Blocklists (licence GPL-3.0). They're maintained daily, come in several strengths, and are published in RPZ, the format Recursor reads. They come as is: the author gives no guarantee for the sources the lists are built from. This chapter uses two:

  • Multi PRO (pro.txt): trackers, ads, telemetry. About 200,000 names, each listed twice (the name and everything under it), so the script below counts about 400,000 entries. A good balance between blocking and breaking.
  • Threat Intelligence Feeds mini (tif.mini.txt): malware, phishing and scam sites.

Alternatives: OISD breaks fewer sites and blocks less; StevenBlack is a single list without strengths. If you want stronger blocking, HaGeZi's PRO++ (pro.plus.txt) blocks more and breaks more.

Be a good guest. The author asks for downloads once or twice a day at most, from GitHub, jsDelivr or their own mirror, and for conditional requests, so an unchanged list isn't downloaded again. The script below does that.

What you need

  • The resolver from chapter 4, running.
  • About 30 minutes (our estimate).

Steps

All on the server.

1. An allow list, so your own names are never blocked

Block lists sometimes contain names you need. Names in this list always get a real answer, whatever the block lists say.

sudo install -d -m 0755 /etc/powerdns/rpz
sudo nano /etc/powerdns/rpz/allow.rpz

Paste, with your domain:

$ORIGIN rpz.allow.
$TTL 300
@                  IN SOA localhost. hostmaster.localhost. 1 3600 600 86400 300
@                  IN NS  localhost.
<your-domain>      IN CNAME rpz-passthru.
*.<your-domain>    IN CNAME rpz-passthru.
  • The first four lines are the header every RPZ file needs.
  • CNAME rpz-passthru. means "answer normally". The line with *. covers everything under your domain, including dns.<your-domain>.

Add a line pair for any other name that a block list breaks for you.

2. Which lists to fetch

sudo install -d -m 0755 /etc/dns-lists /var/lib/dns-lists
sudo nano /etc/dns-lists/lists.conf

Paste:

# <name> <address> [minimum entries, default 1000]
base    https://cdn.jsdelivr.net/gh/hagezi/dns-blocklists@latest/rpz/pro.txt
threat  https://cdn.jsdelivr.net/gh/hagezi/dns-blocklists@latest/rpz/tif.mini.txt

3. The update script

It downloads each list, checks it, and only then swaps it in. A list that fails a check is skipped, and the previous version stays in use.

sudo nano /usr/local/sbin/dns-lists-update

Paste:

#!/bin/bash
set -uo pipefail
CONF=/etc/dns-lists/lists.conf
DIR=/etc/powerdns/rpz
STATE=/var/lib/dns-lists
MIN_DEFAULT=1000          # a real list is never this small; small lists set their own minimum
MAX_SHRINK_PCT=30         # refuse a list that lost more than 30 % of its entries since last time
ENTRY_RE='^[^;$@ ][^ ]*[[:space:]]+(IN[[:space:]]+)?CNAME[[:space:]]+\.'
ETAG_RE='^(W/)?"[A-Za-z0-9._~:/+=-]*"$'   # what we store and send back; anything else is not kept
ETAG_MAX_MIN=9360                         # an ETag is sent for at most 6.5 days after the last full download
PENDING="$STATE/reload-pending"           # set before a list is swapped in, cleared by a successful reload
mkdir -p "$STATE"
save_etag() {
  if [[ "$2" =~ $ETAG_RE ]]; then echo "$2" > "$STATE/$1.etag"
  else rm -f "$STATE/$1.etag"; echo "$1: no usable ETag, the next download is a full one"; fi
}
changed=0; failed=0
while read -r name url min; do
  [[ -z "$name" || "$name" == \#* ]] && continue
  # Conditional only when the file in use is the one the stored ETag belongs to (its sha256 is written last),
  # and the last full download is less than 6.5 days old.
  cur=""; [ -s "$DIR/$name.rpz" ] && cur=$(sha256sum < "$DIR/$name.rpz" | cut -c1-64)
  cond=()
  etag=$(cat "$STATE/$name.etag" 2>/dev/null)
  if [[ "$etag" =~ $ETAG_RE ]] && [ -n "$cur" ] && [ "$cur" = "$(cat "$STATE/$name.sha256" 2>/dev/null)" ] \
     && [ -n "$(find "$STATE/$name.etag" -mmin -$ETAG_MAX_MIN 2>/dev/null)" ]; then
    cond=(-H "If-None-Match: $etag")
  fi
  tmp=$(mktemp "$DIR/.$name.XXXXXX")
  if ! out=$(curl -fsS --proto '=https' --tlsv1.2 --max-time 300 --max-filesize 314572800 \
        -A "dns-lists" ${cond[@]+"${cond[@]}"} -o "$tmp" -w '%{http_code} %header{etag}' "$url"); then
    echo "$name: download failed"; rm -f "$tmp"; failed=1; continue
  fi
  code=${out%% *}; new_etag=${out#* }
  if [ "$code" = 304 ] && [ ${#cond[@]} -gt 0 ]; then
    # keeps the shrink check if the count was lost
    [ -s "$STATE/$name.count" ] || grep -cE "$ENTRY_RE" "$DIR/$name.rpz" > "$STATE/$name.count"
    echo "$name: not modified"; rm -f "$tmp"; continue
  fi
  if [ "$code" != 200 ]; then
    echo "$name: unexpected HTTP $code"; rm -f "$tmp"; failed=1; continue
  fi
  entries=$(grep -cE "$ENTRY_RE" "$tmp")
  soa=$(grep -cE '[[:space:]]SOA[[:space:]]' "$tmp")
  prev=$(cat "$STATE/$name.count" 2>/dev/null || echo 0)
  if [ "$soa" -lt 1 ] || [ "$entries" -lt "${min:-$MIN_DEFAULT}" ]; then
    echo "$name: rejected (SOA=$soa, entries=$entries)"; rm -f "$tmp"; failed=1; continue
  fi
  if [ "$prev" -gt 0 ] && [ $(( entries * 100 / prev )) -lt $(( 100 - MAX_SHRINK_PCT )) ]; then
    echo "$name: rejected, shrank from $prev to $entries entries"; rm -f "$tmp"; failed=1; continue
  fi
  # The ETag is kept only once the file it belongs to is the one in use; the sha256 is written last.
  new=$(sha256sum "$tmp" | cut -c1-64)
  if [ "$new" = "$cur" ]; then
    echo "$entries" > "$STATE/$name.count"; save_etag "$name" "$new_etag"; echo "$new" > "$STATE/$name.sha256"
    echo "$name: unchanged ($entries entries)"; rm -f "$tmp"; continue
  fi
  touch "$PENDING"   # the recursor must load the new file even if this run stops before the reload
  chmod 0644 "$tmp"; mv -f "$tmp" "$DIR/$name.rpz" || { echo "$name: could not swap in"; rm -f "$tmp"; failed=1; continue; }
  echo "$entries" > "$STATE/$name.count"; save_etag "$name" "$new_etag"; echo "$new" > "$STATE/$name.sha256"
  echo "$name: updated, $entries entries (was $prev)"; changed=1
done < "$CONF"
if [ "$changed" = 1 ] || [ -e "$PENDING" ]; then
  # RPZ files are part of the "Lua config" clauses of the YAML settings. Loading them can take longer
  # than rec_control's default wait of 5 s on a small server, hence --timeout.
  if rec_control --timeout=120 reload-lua-config >/dev/null; then
    rm -f "$PENDING"; echo "recursor: lists reloaded"
  else
    touch "$PENDING"; echo "recursor: reload FAILED (tried again on the next run)"; failed=1
  fi
fi
exit $failed

What it does, for each list:

  • Asks first whether the list changed. It sends the list's last version tag (ETag), and an unchanged list answers "not modified" without being downloaded again, as the list author asks. At least once a week (every 6.5 days) it downloads each list in full anyway.
  • HTTPS only (--proto '=https'), at most 5 minutes and 300 MB per download.
  • It looks like a list: it has an RPZ header (SOA) and at least 1,000 entries.
  • It didn't shrink by more than 30 % since last time. A list that suddenly loses a third of its names is more likely broken than cleaned up.
  • Unchanged lists are left alone (same checksum).
  • mv -f swaps the file in one step, so Recursor never reads a half-written list.
  • Only if something changed, Recursor reloads its lists, without a restart. It may take longer than the usual 5 seconds on a small server, so the script waits up to 2 minutes (--timeout=120).
  • A reload that fails is tried again on the next run. A marker file (reload-pending) is set before a new list is swapped in, and removed only after a successful reload.
  • The version tag is stored only once the list it belongs to is the one in use, so a half-finished run can never make the script skip a list it doesn't have.

Make it executable, and run it once:

sudo chmod 0755 /usr/local/sbin/dns-lists-update
sudo /usr/local/sbin/dns-lists-update

It shows base: updated, … entries (was 0) and the same for threat. The reload line may fail this first time, because Recursor doesn't know the lists yet; that's fine, and the next step restarts it.

4. Tell Recursor about the lists

sudo nano /etc/powerdns/recursor.d/dns.yml

Under recursor:, below stats_ringbuffer_entries: 0, add:

  rpzs:
    - name: /etc/powerdns/rpz/allow.rpz
      policyName: allow
    - name: /etc/powerdns/rpz/base.rpz
      policyName: base
    - name: /etc/powerdns/rpz/threat.rpz
      policyName: threat

The order matters: the first list that matches a name decides. The allow list comes first, so it always wins.

Before restarting, make sure all three files are there:

ls -l /etc/powerdns/rpz/allow.rpz /etc/powerdns/rpz/base.rpz /etc/powerdns/rpz/threat.rpz

Don't skip this. Recursor starts even when a list is missing or broken. It just runs without it, and blocks less than you think.

Restart Recursor:

sudo systemctl restart pdns-recursor

5. Update every day

Two small files tell the server to run the script daily.

sudo nano /etc/systemd/system/dns-lists.service
[Unit]
Description=Update resolver block lists
After=network-online.target pdns-recursor.service
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/dns-lists-update
Nice=10
sudo nano /etc/systemd/system/dns-lists.timer
[Unit]
Description=Daily block list update

[Timer]
OnCalendar=*-*-* 12:15:00 UTC
RandomizedDelaySec=45m
Persistent=true

[Install]
WantedBy=timers.target
  • HaGeZi publishes once a day at 11:00 UTC; this runs after that, at a random minute within 45 minutes.
  • Persistent=true: if the server was off at that time, it runs as soon as it's back.

Switch it on:

sudo systemctl daemon-reload
sudo systemctl enable --now dns-lists.timer
systemctl list-timers dns-lists.timer

The last command shows the next run.

How you know it worked

All on the server:

  1. A tracker doesn't exist any more:

    kdig @127.0.0.1 -p 5353 google-analytics.com A

    shows status: NXDOMAIN.

  2. Normal sites still work:

    kdig @127.0.0.1 -p 5353 www.wikipedia.org A

    shows status: NOERROR and an address.

  3. Your own domain is never blocked:

    kdig @127.0.0.1 -p 5353 dns.<your-domain> A

    shows your server's address.

  4. Recursor loaded all three lists, without errors:

    sudo journalctl -u pdns-recursor --since "-10 min" | grep -c "Done loading RPZ from file"
sudo journalctl -u pdns-recursor --since "-10 min" | grep -c "Exception while loading RPZ"

    The first shows 3, the second 0. The wording comes from Recursor's code and may change between versions.

  5. Memory is fine: ps -o rss=,comm= -C pdns_recursor stays well under the server's memory. In our setup, with more lists than this chapter uses, Recursor went from 37 MB to about 190 MB.

Pitfalls

  • Reloading lists with the YAML settings. Lists are reloaded with rec_control reload-lua-config, not reload-zones. The name is confusing; we tried the wrong one first.
  • A browser that knows the old answer. Devices remember answers for a while. When you test a newly blocked name, use one the device has never looked up, or wait.
  • Some sites break. When something you need stops working, add it to the allow list (step 1), then sudo systemctl restart pdns-recursor. If you think the list is wrong, tell the author: report the domain, the list it's on, and in a short note what breaks.
  • A missing list doesn't stop Recursor. It logs an error and runs without that list. Run check 4 after every change.
  • What DNS blocking can't do. An app that already knows its servers' addresses doesn't need to look up a name. In our test, the Instagram app kept working while facebook.com was blocked in Safari. Test apps, not only websites.

Time and cost

  • About 30 minutes (our estimate).
  • Free. Asked about support, the list author said no donations; good reports of wrong blocks help more.

Previous: Chapter 5: Certificates · Next: Chapter 7: Logs that forget

Text: CC BY-SA 4.0 · Commands and configs: CC0