My FiOS setup at home is a little bit non-standard. Like many network geeks, I want a little bit more than the Verizon-provided Actiontec router will provide, such as multiple vlans, bigger subnets, an IPv6 tunnel, etc. I run the Actiontec behind a Ubiquiti EdgeRouter Lite, which speaks Ethernet to the Verizon-provided BPON ONT on the wall. In a nutshell, the STBs are double-natted.

Believe it or not, this mostly works. I don't quite have the port forwarding, passthrough, etc. working perfectly... for general use everything works on my STBs except for remote programming of recordings with a web browser, but that's no great loss - have you seen Verizon's interface for doing that? Really.

The other corner case besides remote programming of the DVR is that, in order to provision a new STB one has to put things back the way Verizon had them originally, which means a recable. As luck would have it, our FiOS Quantum whole-house DVR box has been getting exponentially more flakey, rebooting at odd intervals, having the tuner/decoder freeze, etc. The last time I did this recabling dance was literally years ago, so it was doomed to be a bit of a goat rope.

You know you've been working in Big Cable too long when you assume that the behavior of any vendor-provided CPE is going to be more or less cablemodem-like. That's definitely not the case for FiOS.

I'm paying for residential service here. In both cable-land and FiOS-land, this means that the system is set up to only allow a single MAC address on the CPE end of things, so you can only DHCP for a single (IPv4) address, only have a single NAT router, etc.

The way it's implemented, though, is different. In DOCSIS, the cablemodem typically gets in on the game and installs a filter that only allows the first MAC address it hears to talk back and forth. In PON land, though, it seems to be implemented in the DHCP server. Power cycling the ONT (which requires disconnecting the battery backup) has no effect.

Andrew says he suspects that rather than being in the ONT, the Verizon DHCP infrastructure has some policy that enforces one address per interface, probably associated by some data provided in the relay agent option. This pretty much matches what I was thinking.

Confirmed by the folks at DSLReports, it's necessary to release the lease on the address before trying again with a different MAC address. Link contains how to do this on the Actiontec. Because I was lazy (and the CLI's differences from VyOS are poorly documented and not all in one place), I leaned on Clayton to dig in his notes and tell me that on the ERL the command is

{% raw %} release dhcp interface

You don't necessarily have to do this though. Both the Actiontec and the ERL are capable of doing MAC address spoofing. If you're like me and have years of dissatisfied history with IP networking on Linux, you'll be concerned that the Linux stack on the Actiontec would react poorly to seeing an address that it thinks is its own on another interface (not the one that it has that MAC address on). Clayton assured me that the Linux stack has improved greatly in recent years and on the ERL it actually works fine. And indeed it does, at least on EdgeOS v1.8.5 which is what I'm running right now.

If you want to have the external interface of your EdgeRouter spoof the built-in MAC address of the Actiontec (likely a wise move, though the Actiontec is capable of spoofing too, and right now we're doing spoofing on both the ERL and the Actiontec of the MAC address of a downrev-and-long-since-returned Actiontec), the command for that is

{% raw %} set interfaces ethernet eth0 mac '00:26:ca:fe:ba:be'

Once you're set up to have the same external facing MAC address on your Actiontec and your ERL, you can swap them back and forth more or less with impunity (possibly needing a reboot of whoever's currently hooked to the ONT so that it renews its lease.

If you see the Actiontec flopping back and forth between trying PPPoE and DHCP, that's normal - it will try both since they use the same router for DSL - but it means it's not getting a reply to the DHCP request.