A couple of people asked me how that dns-nameservers thing came to my attention.
After all, if I'm running a nameserver on 127.0.0.1 (I am) and the machine comes up all the way (it does), then everything should be copacetic and I would never notice, right?
I do DNSSEC validation as a matter of course. This is a matter of eating my own dog food; I am one of the people who signs the DNSSEC root. I'll admit to being a little bit slack about getting off my butt to sign my own stuff, but if someone cares enough to sign their records I absolutely want to honor that. Besides, most of the TLDs are signed. And that's where things get interesting.
I've been enamored of little ARM-based computers lately. No, not the ones that are super popular and cost less than $40, but rather the ones with four times the cores (clocked at > 2x the speed) and 4x the RAM, have a little fewer GPIOs, and cost less than $70. They even have a real time clock on them, something that the popular cheapie does not. They make nice little DNS and DHCP servers that remain up even when the virtualization platform in the house is down for maintenance (rare, but it's happened).
Unfortunately, if it's going to hold time over a reboot or power cycle it needs an external battery, which is a <a href=http://ameridroid.com/products/rtc-battery$2.95 accessory. I had Gaige order up one of these little computers to use at home and forgot to include the battery on the BOM. No problem; I figured we'd just get time from the network at boot time.
Now, the ZSKs (zone signing keys) that we generate for the Root Zone Operator during the DNSSEC ceremony are only good for a month each. That means that your clock needs to be kinda-sorta-right, at least within a month, if you are to be able to validate recursing through any TLD that is signed. When the computer boots, it thinks that the time is sometime in 2000 or so if it doesn't have a carry-over battery from the last time it set the time. And the NTP servers are listed in /etc/ntp.conf as .ubuntu.pool.ntp.org.
When a recursing DNS server that is trying to validate discovers apparently-bogus data, it returns SERVFAIL. The idea all along was to only hand out the address of the local DNS server to DHCP clients, so I figured that a good workaround would be to additional lines in resolv.conf so that upon getting the SERVFAIL, the local resolver library should discover the other nameservers and get unverified data from in order to set the time.
No such luck. Only 127.0.0.1 makes it in if it's first.