It’s long been my intention to write up a little bit here about CARP and how it can interfere with VRRP - a trap for the unwary. Without getting into a lot of rehashing of who did what when, the net result is that the OpenBSD folks decided to use IANA’s Ethernet OUI rather than applying for their own and managed by chance (of 1 in 65535) or design to collide with reserved MAC addresses that were already assigned to a draft version of VRRP (as well as recycling an IP protocol number, but that’s a topic for another day).
I managed to get bitten by this myself at a client’s site a couple of years ago - neither he nor I knew at the time of the likelihood for interference between the two when the same virtual router ID is configured in both protocols. That was pretty embarrassing.
Procrastination can be its own reward though, and Paul Wall managed to beat me to the punch by posting a deep dive on this problem (with a few acrimonious words on the side) to the <a NANOG mailing list. So without further ado, here’s Paul’s post, which does a better job covering the issue than I could have left to my own devices. Thanks Paul!
From: Paul WALL <pauldotwall () gmail com> Date: Tue, 22 Apr 2014 13:30:32 -0400 On Tuesday, April 22, 2014, Henning Brauer <hb-nanog () bsws de> wrote: > I won't waste time on your uninformed ramblings, you have the facts > plain wrong. There is enough material on the net for everybody to read > up on what happened. > > "carp causing outages" however is nothing short of a lie. carp > announces itself as vrrp version 3. anything trying to parse it as > vrrp2 without checking the version number in the header is buggy as > hell and that is ITS fault, not carps. affected cisco 3600, afair. I wasn't talking about CARP's announcements causing outages due to bugs in VRRP implementations, I was talking about CARP's intentional use of another organization's OUI and deliberately constructing its bits in the host section to conflict with established practice for VRRP. That was childish, and causes outages. This design choice should be documented in CARP's man page. It is not. In response to Ryan Shea, here's the way it breaks down: Both CARP and VRRP use virtual router MAC addresses that start with 00:00:5e. This organizational unique identifier (OUI) is assigned to IANA, not OpenBSD or a related project. The CARP authors could have gotten their own from IEEE. OUIs are not free but the cost is quite reasonable (and was even more reasonable years ago when this unfortunate decision was made). The next two octets for IPv4 VRRP are 00:01. Highly coincidentally, the CARP folks *also* decided to use 00:01 after they got upset at the IETF for dissing their slide deck. If either of these decisions had not been made, we would not be having this discussion today. The last octet is the VRID for both CARP and VRRP. If you don't have the same VRID configured, the protocols should peacefully coexist, assuming no bugs in the software listening to the multicast advertisements (which Henning mentioned above - a legitimate concern to be sure, but not the big one). So yes, the problem only exists if you are running VRRP and CARP on the same subnet (say, a pair of routers speaking VRRP and a pair of firewalls backing each other with CARP and pfsync, which is actually fairly common) and happen to have a host identifier conflict. In a completely random world the likelihood of this is 1 in 256. Unfortunately, human nature and a plethora of examples on the intarwebs makes "interface GigabitEthernet 0/3 // vrrp 1 ip blah" reasonably likely. The same human nature causes the out of the box configuration on many products, e.g. pfSense, to be "ifconfig carp0 vhid 1". Presto - outage for everyone on the subnet. Soapbox time. There are some people who decide that they will only run FOSS software because of how they feel about software patents. In my case, I will not run CARP because of how I feel about folks deliberately violating the interoperability maxim ("be conservative in what you send and liberal in what you accept") and causing *me* to be the collateral damage. I think we all have enough on our plates dealing with legitimate software bugs without having rogue protocols deliberately interfering with our networks because of a grudge. Is CARP malware? A trojan? I'm not sure I'd go that far, but it seems to meet some of the definitions. Nothing personal Henning (and I like what you did with OpenBGPd and OpenNTPd) but you'd gain a lot of respect in my eyes, as well as a bunch of other people's, if you publicly admitted the CARP OUI decision was a huge mistake. If your lawyers have advised you not to apologize because of liability concerns (despite that "no warranty" bit in the BSD license) it's OK - I completely understand. Drive Slow, Paul