DuckCorp » DuckCorp Infrastructure

{{toc}}

h1. DNS

h2. Zone Management

h3. Adding a Zone

The zones are declared in _host_vars/<dnsserver>/dns.yml_ and the playbook _playbooks/tenants/duckcorp/dns.yml_ is in charge of updating all configurations (see the _bind9_ role documentation to understand the parameters).

As for the zone content:

* the preferred method is though the _dns4tenants_ method; in the same config above you can define git repositories for tenants containing their zones which are then automagically updated on the server when the git repo is updated (max 5min delay) (see the _dns4tenants_ role documentation to understand the parameters)

* *OBSOLETE* Banya service using GPG emails to update the zone; users had much problems with their MUA and the maintenance of the Ruby code was also difficult, then we are migrating them to the _dns4tenants_ method (WIP)

* *OBSOLETE* zone files directly written on the server in _/etc/bind/masters/_ (beware of ownership _root:bind_); there is no history and it's prone to mistake, then we are migrating them to the _dns4tenants_ method (WIP)

If the zone is DNSSEC-signed, the publishing of keys in the parent zone may need to be done manually; more details below.

h3. Updating a Zone

h4. dns4tenants method

You just need to push changed in the correspondign git repo holding the zone. You don't need to care about checking, reloading or DNSSEC, all of this is done for you. It can be useful to add checks in the repo's CI though to catch errors early. In case of error the tenant will be notified by mail.

h4. Manual method

Edit the file _/etc/bind/masters/<zone-name>.zone_ on the primary master (Orfeo for all zones except DuckLand zones using Elwing).

Do not forget to update the serial!

Better to check the file validity before publishing the zone:

<pre>

named-checkzone <zone-name> <zone-file>

</pre>

Then to publish the zone (DNSSEC-signed zones too):

<pre>

rndc reload <zone-name>

</pre>

h3. Reseting the Zone Serial

The serial needs to be increased by steps, as described "in this article":http://www.microhowto.info/howto/reset_the_serial_number_of_a_dns_zone.html.

h2. Secure Zone Transfers

To secure zone transfers, a TSIG key needs to be created and added on both sides. Beware the key name *must* be identical on both side. 

DNS server groups (servers allowed to request transfer) and keys can be defined in _host_vars/<dnsserver>/dns.yml_ and _host_vars/<dnsserver>/dns.vault.yml_ respectively. If they are to be used on all servers, then you can declare them in _group_vars/dns_servers/dns.yml_ and _group_vars/dns_servers/dns.vault.yml_ respectively.

You can a new key using:

<pre>

dnssec-keygen -a HMAC-SHA512 -b 512 -n HOST taiste

</pre>

Take the 'Key' part in 'Ktaiste.*.private' file, to put into the configuration.

The same playbook (_playbooks/tenants/duckcorp/dns.yml_) is used to update the configuration.

h2. DNSSEC

Here are notes about using Bind KASP (version 9.16 required).

All general info above about DNSSEC does not change, especially the rollover steps are similar even if the tooling change, and testing the zone is identical.

The Ansible _bind_ role has been updated in a branch to be able to use Bind KASP for DNSSEC. Our Ansible repository now tracks this branch and add the necessary parameters to use it. Please look at the role's documentation to understand the inner technical details, this page is about administration of the solution.

h3. Introduction

Better read some documentation before fiddling with the controls:

* "Bind DNSSEC Guide":https://downloads.isc.org/isc/dnssec-guide/html/dnssec-guide.html (for general principles)

* "KSK Rollover":https://blog.webernetz.net/dnssec-ksk-key-rollover/ (key is created by KASP instead of using `dnssec-keymgr` but it's a good example)

* "Bind KASP (aka dnssec-policy)":https://kb.isc.org/v1/docs/en/dnssec-key-and-signing-policy (Bind9's new system)

* "Paper KASP is based on":https://nlnetlabs.nl/downloads/publications/satin2012-Schaeffer.pdf (explains the state machine)

* "Future KSK Rollover Automation":https://www.dns.cam.ac.uk/news/2019-01-30-rollover.html

* "CDS/CDNSKEY (RFC 7344) automation":https://jpmens.net/2017/09/21/parents-children-cds-cdnskey-records-and-dnssec-cds/

Key materials are created on-demand by Bind using the policy parameters, so no need to do anything outside Ansible configuration of the zones and policy parameters. Cleanup of old keys when they become obsolete or when a zone is removed is not yet done though.

h3. Notes about migrating from a previous version (historical)

It is important to cleanup old keys first if switching from dnssec-keymgr to dnssec-policy or old keys would get in the way.

More generally look at "this ticket tracking the problems we encountered":https://projects.duckcorp.org/issues/720

As Bind is not using the usual date-based zone serial, it can be less misleading to reset the serial before migration (see dedicated chapter above).

h3. Zone Status

General zone info, including the real published serial (after signing, resigning if it happens, rollovers…) and planned signing events:

<pre>

rndc zonestatus <zone-name>

</pre>

h4. Zone Keys

To know which keys (<key-id>) are currently signing a zone (may be inactive and not deleted yet):

<pre>

rndc dnssec -status <zone-name>

</pre>

Keys are stored in _/etc/bind/keys_ and you can use the key ID to locate the corresponding file this way:

<pre>

ls /etc/bind/keys/K<zone-name>.+*+*<key-id>.key

</pre>

Inside you can read the key type (KSK/ZSK) and the lifetime schedule (so important rollover dates).

The KASP is in charge of the key maintenance according to the policy. It is possible to alter the timing using the _dnssec-settime_ tool in case of bugs but that should not be needed.

In this case, after doing the modifications, Bind needs to be notified using:

<pre>

rndc loadkeys <zone-name>

</pre>

h4. Parent Zone Publishing

To see if the zone KSK keys are properly published in the parent zone:

<pre>

dnssec-checkds <zone-name>

</pre>

h3. Key Rollover

To get a view of the schedule:

<pre>

rndc dnssec -status <zone-name>

</pre>

To have a list of KSK keys that needs publishing on the parent zone:

<pre>

dnssec-checkds <zone-name>

</pre>

The ZSK key rollover is handled automatically by Bind (KASP), so admins have nothing to do.

The KSK rollover implies contact with the parent zone:

* if we do not manage the parent zone:

** if the parent zone handles CDS/CDNSKEY (RFC 7344) then it will grab the new DS automagically (but most TLDs do not support it yet)

** else a manual step to get the DS entry in their zone is needed (manually in their UI, maybe via an API)

* if we manage the parent zone:

** if the zone publishes the CDS/CDNSKEY RRs (all our zones have them) then we simply need to define the list of _dnssec_children_ in the parent zone configuration (see bind9 role documentation) and a script will make the update

** else we need to organize with the tenant on a method to exchange the DS

Then Bind needs to be informed that a new KSK is properly published in the parent zone and an obsolete KSK had been removed from the parent zone (this is unfortunately not automatic yet):

<pre>

rndc dnssec -checkds -key <new-key-id> published <zone-name>

rndc dnssec -checkds -key <old-key-id> withdrawn <zone-name>

</pre>

(each can be done anytime but the rollover will be completed only when both are done)

h3. KSK Rollover Workflow

We use the Double-Signature method with some overlap of DS publishing.

Bind's KASP uses a new system of states which does not represent steps and are not documented yet. We decided to keep the usual state names to reflect the states and try to map to the new system when we have the knowledge.

Here are the states and what needs to be done:

* *created* state:

** new created key (new zone or key replacement), this key is not used yet

** action: wait

* *publish* state:

** the key is added to the zone and used to sign but not yet published in the parent zone

** KASP says the key's DS is _rumoured_

** action: if not automated, wait for propagation, export the key (type depending on the registrar) and add it to the parent zone (Web UI, API…):

*** DS: in digest format using the *dnssec-dsfromkey -2 <ksk-filename>* command (see previous chapter to get the absolute filename for the current KSK key, any of the _key_ or _private_ file would do)

*** full key: in _<ksk-filename>_ the _<key>_ is on a line formated like _<zone-name> <ttl>  IN DNSKEY <flags> <protocol> <algorithm> <key>_ (<flags>, <protocol> and <algorithm> are three numbers, the rest is the <key>; you can copy it with the spaces)

** action: after the DS TTL has passed, check if it is well published (*dnssec-checkds <zone-name>*) and notify Bind (see *rndc dnssec -checkds* above)

* *active* state:

** the key is used to sign and published in the parent zone

** KASP says all states to _omnipresent_

** action: wait for the next rollover

* *unpublish* state:

** the key is still used to sign but should top being used soon

** KASP says the key's goal is _hidden_ and DS is _unretentive_

** action: if not automated, remove the DS key from the parent zone

** action: after the DS TTL has passed, check if it is no longer published (*dnssec-checkds <zone-name>*) and notify Bind (see *rndc dnssec -checkds* above)

* *inactive* state:

** the key is no longer used to sign nor published in the parent zone

** action: remove the key materials (since Bind does not cleanup automatically yet)

Currently we need to check manually when to do the KSK rollover. The coverage command above and _next key event_ in the zone info should help build a little script to warn us in time (the old one for dnssec-keymgr cannot be used anymore). The system will wait for us even if the expected date has passed.

Starting March 2023 we are not using the parental-agents Bind9 feature which checks the status of the parent zone. For parent zones we manage we should not need to do anything anymore. For parent zones which are TLDs we still need to add the new key and later remove the old but the system should detect when the key is published and withdrawn automagically.

h3. Old keys cleanup

Old keys are moved to the _/etc/bind/keys/attic/_ manually at the moment. Before moving away the files we need to wait for the key status (_rndc dnssec -status <domain>_) to be:

* published: no

* key signing: no

* goal: hidden

* dnskey: hidden

* ds: hidden

* key rrsig: hidden

Then we can restart the daemon.

h3. Checking a Zone

Test a Zone using a DNSSEC-enabled resolver:

<pre>

dig <zone-name> +dnssec

</pre>

You need to get the ad flag. If you get the aa flag, then you're interrogating one of the official NS for the zone, then try on another server to be sure your configuration is OK (remotely with *@<server>* as first command option).

Test a Zone using an external web tool:

* http://dnssec-debugger.verisignlabs.com/

* http://dnsviz.net/

h3. Forcing a policy change to be applied at once

Via Ansible it is possible to change the policy directly and Bind should trigger the changes automagically. Currently we have not tested a change of policy with KASP.yet. We would like to test an algorithm rollover but we're waiting for some other bugs to be fixed first.

h3. Unsecuring a Zone

First the DS needs to be removed from the parent zone, then we need to wait for the DS TTL to expire (and it's probably better to wait a few days for Inetrnet caches to expire) before unsigning (which can be done by changing the zone's _dnssec_policy_ to _insecure_ in the Ansible configuration). It has not been tested yet since we never had the need.

Key materials need to be removed manually.

h3. Forcing an Early Rollover

It is possible to do so: https://blog.webernetz.net/dnssec-ksk-emergency-rollover/

You can trigger an immediate change of KSK (with <key-id> the ID of the key you wish to replace):

<pre>

rndc dnssec -rollover -key <key-id> <zone-name>

</pre>

There is currently a bug so it may take up to a week to trigger.

h2. Checking Servers

* "ISC EDNS Compliance Tester":https://ednscomp.isc.org/ednscomp/

h3. DNSSEC Checks

Should return a A record and have the *ad* flag set:

<pre>

dig sigok.verteiltesysteme.net @127.0.0.1

</pre>

Should return *SERVFAIL*:

<pre>

dig sigfail.verteiltesysteme.net @127.0.0.1

</pre>

h2. Problems

h3. receive_secure_serial: not exact

This means the inline-signing journal is corrupted and changes to the zone cannot be applied to the signed zoned.

Workaround:

<pre>

rndc sync -clean <zone>

rndc stop

</pre>

then bump the zone's serial and restart Bind, it should have solved the problem.

If this does not work:

<pre>

systemctl stop bind9

rm /var/cache/bind/masters/<zone>.zone.*

systemctl start bind9

</pre>