Many years ago, when I was but an infant, the first computers were connected on the ARPANET - the seminal computer network that would eventually evolve to become the Internet. Computers at the time were large and expensive; indeed the first version of NCP - the predecessor of TCP/IP - only countenanced roughly 250 computers on the network.

The name (human friendly) to network address (computer friendly) mapping on this network was maintained via a "hosts file" - literally a flat file of ordered pairs, creating the connection between host (computer) name and address.

So it continued as computers got less expensive and proliferated, the Network Effect caused more institutions to want to be connected to the ARPANET. TCP/IP was developed in response to this, with support for orders of magnitude more connected computers. Along the way, the military users of the network got carved off into its own network, and by the early 1980s we had the beginnings of the Internet, or a "catenet" as it was sometimes called at the time - a network of networks.

Clearly, as we went from "a couple hundred computers" to "capacity for billions", a centrally managed host file wasn't going to scale, and by the early 1980s development had started on a distributed database to replace the centrally managed file. The name for this distributed database was the Domain Name System, or DNS.

It's important to realize that at the time, access to the network of networks was still restricted to a chosen few - higher education, research institutions, military organizations and the military-industrial complex (ARPA, later DARPA, was, after all, an activity of the United States Department of Defense), and a few companies that were tightly associated with one or more of those constituencies. Broad public commercial access to the Internet was many years in the future.

It was in this environment that the DNS sprang forth. Academics, military researchers, university students - a pretty collegial environment. Not to mention paleo-cybersecurity practices - indeed the word "cybersecurity" may not have even been coined yet, though the notion of "computer security" dates back to the early 1970s.

I've mentioned this brief "history of the early Internet" to preemptively answer the question which inevitably arises: why didn't DNS have better security built in? The answer is twofold: firstly it didn't have to based on the environment that it evolved in, and secondly, even if it had, the security practices would have been firmly rooted in 1980s best practices, which would certainly be inadequate by modern standards.

Discovery of security flaws in 1990 led the IETF to begin development on Domain Name System Security Extensions (DNSSEC) in 1995. Early versions were difficult to deploy. Later versions improved somewhat. But inertia is a thing, the status quo tends to prevail, and there was very real concern that DNSSEC would be a net reliability minus (security vs. availability can be a tricky circle to square), concentrate power in undesirable ways, and result in other unforeseen negative effects.

At the end of the day, as it so often does, it took a crisis to get the ball rolling for real. In 2008, Dan Kaminsky discovered a fundamental flaw in DNS, which simplified cache poisoning - essentially making it possible for an attacker to misdirect users to arbitrary web sites.

In less than two years, the DNS root would be cryptographically signed - allowing those who wished to sign their domains as well to create a cryptographic chain of trust authenticating their DNS lookups. This is non-repudiation, not non-disclosure - DNS queries and responses continued to happen in the clear. But this time, responses came back with a digital signature, courtesy of DNSSEC.

David Huberman at ICANN did a splendid slide deck explaining how it all works.

Trust in a system requires more than technical correctness. It involves trust in the execution of running the system itself. For that reason ICANN decided that it would build a framework to facilitate trust and transparency. Among other things it included:

• Placing the cryptographic material in two highly secure sites, one near Washington DC and one in Los Angeles (geographic diversity)
• Creating a multi-layered security regimen requiring several people to access the Key Management Facility
• Storing cryptographic material in offline HSMs which utilize Shamir's Secret Sharing to require a quorum of at least 3 out of 7 Crypto Officers to be present in order to "wake them up"
• Trusted Community Representatives with roles of Crypto Officer and Recovery Key Share Holder
• Highly scripted (and therefore auditable) ceremonies surrounding handling the cryptographc material
• Live streaming all events
• Hosting External Witnesses from the community who have expressed interest in being present for a ceremony in person

When the call for volunteers to be Trusted Community Representatives came out, I was no stranger to community involvement, having served several years on the ARIN Advisory Council and done community work (and later Board work) for NANOG. I was employed by a Top Level Domain operator, and submitted my CV and expressed my interest.

That's how I found myself in Culpeper, Virginia in 2010 as Crypto Officer 4 at the first ceremony for signing the DNSSEC Root. I had no idea that I would still be doing it fifteen years later. I was the last of the original Crypto Officers for KMF-East, and the second last overall - outlasted only by Subramanian "SM" Moonesamy, who is Crypto Officer 7 for KMF-West.

It's been an adventure. I've been a small participant in root key rolls, put in a B-roll appearance on BBC Horizon (S53E13), become friends with many of the people I served with, overseen ceremonies remotely during the COVID lockdown, and witnessed an amazing pivot by ICANN staff who managed to get new HSMs selected, tested, integrated, and deployed on only 8 months' notice, a feat which I remain in awe of.

I was an early advocate for improving trust in our process by leveraging natural turnover and backfilling existing TCRs with people selected from a broader set of qualified individuals than just the fellowship of DNS protocol experts, operators, and developers. I'm grateful that our voices were heard.

On November 13th 2025, I passed the torch to Lodrina Cherne who is now Crypto Officer 4 for KMF-East. Lodrina is a security researcher, an educator with an emphasis on digital forensics, and works in security engineering at a large cloud provider. I'm honored to have her as my successor.

I've had several people reach out to me to ask what prompted me to step back from the ICANN volunteer work. Those who were hoping for some kind of salacious dirt or scandal were sorely disappointed - quite the opposite, this is a huge success story and I'm pleased to have been able to do my small part. A direct cut and paste from Slack logs with one of them follows:

What led you to step back from ICANN?

Several things:

• It was understood to be a 5 year commitment. I've been doing it for more than 15.
• It was broadly agreed among the cohort many years ago (over a decade ago) that more people from more diverse backgrounds than just DNS-old-boy-network (which was the original group of TCRs) was a Good Thing.
• Many people cycled out earlier; I was happy to let the folks for whom travel was more odious go first. But it's only practical and only really good for the system to cycle out a single TCR per ceremony.
• COVID delayed this. Kaminsky's untimely death and subsequent replacement as a recovery key shareholder (RKSH) delayed this.
• A further delay was the AEP Keyper HSM getting abruptly EOLed, and the transition to the Thales Luna HSMs. It went off without a hitch after being researched, developed, and executed in 8 months - a record which I stand in awe of and which is a true testament to the skill and commitment of the ICANN PTI team. ICANN expressed the desire for continuity among long-serving COs past that milestone; Frederico Neves (fellow original Crypto Officer) and I were willing to extend our stay for that.

So in short it was time to pass the torch. Everyone has been doing everything right. I remarked at the end of the Ceremony 59 that when we started doing this 15 years ago, success was not guaranteed; it took the Kaminsky bug to get us over the line to actually deploy it. Today, the major Unix DNS resolvers ship with DNSSEC validation enabled. All of the major public DNS resolvers (google, quad9, cloudflare) do DNSSEC validation. I thanked everyone who has been responsible for and put their personal credibility on the line for the security, integrity, and credibility of this process and stated that I was honored to have been able to play a small part in doing that.

Epilogue:

I won't be participating in most East Coast ceremonies from here on out, but I don't rule out occasionally showing up as an external witness, particularly at KMF-West where I have never visited in person.

Here scans of our ceremony scripts from both Ceremony 59 and the previous day's administrative ceremonies.

Root DNSSEC KSK Ceremony 59

Root DNSSEC KSK Administrative Ceremony 59 Backup HSM Acceptance Testing

Root DNSSEC KSK Administrative Ceremony 59 Safe #2 Credentiais Safe Deposit Box Lock Assembly Programming

kskm-ksrsigner-logs.pdf