- Mon 28 April 2014
- misc
It's long been my intention to write up a little bit here about CARP and how it can interfere with VRRP - a trap for the unwary. Without getting into a lot of rehashing of who did what when, the net result is that the OpenBSD folks decided to use IANA's Ethernet OUI rather than applying for their own and managed by chance (of 1 in 65535) or design to collide with reserved MAC addresses that were already assigned to a draft version of VRRP (as well as recycling an IP protocol number, but that's a topic for another day).
I managed to get bitten by this myself at a client's site a couple of years ago - neither he nor I knew at the time of the likelihood for interference between the two when the same virtual router ID is configured in both protocols. That was pretty embarrassing.
Procrastination can be its own reward though, and Paul Wall managed to beat me to the punch by posting a deep dive on this problem (with a few acrimonious words on the side) to the NANOG mailing list. So without further ado, here's Paul's post, which does a better job covering the issue than I could have left to my own devices. Thanks Paul!
{% raw %}
From: Paul WALL
On Tuesday, April 22, 2014, Henning Brauer <hb-nanog () bsws de> wrote:
> I won't waste time on your uninformed ramblings, you have the facts
> plain wrong. There is enough material on the net for everybody to read
> up on what happened.
>
> "carp causing outages" however is nothing short of a lie. carp
> announces itself as vrrp version 3. anything trying to parse it as
> vrrp2 without checking the version number in the header is buggy as
> hell and that is ITS fault, not carps. affected cisco 3600, afair.
I wasn't talking about CARP's announcements causing outages due to
bugs in VRRP implementations, I was talking about CARP's intentional
use of another organization's OUI and deliberately constructing its
bits in the host section to conflict with established practice for
VRRP. That was childish, and causes outages. This design choice
should be documented in CARP's man page. It is not.
In response to Ryan Shea, here's the way it breaks down:
Both CARP and VRRP use virtual router MAC addresses that start with
00:00:5e. This organizational unique identifier (OUI) is assigned to
IANA, not OpenBSD or a related project. The CARP authors could have
gotten their own from IEEE. OUIs are not free but the cost is quite
reasonable (and was even more reasonable years ago when this
unfortunate decision was made).
The next two octets for IPv4 VRRP are 00:01. Highly coincidentally,
the CARP folks *also* decided to use 00:01 after they got upset at the
IETF for dissing their slide deck.
If either of these decisions had not been made, we would not be having
this discussion today.
The last octet is the VRID for both CARP and VRRP. If you don't have
the same VRID configured, the protocols should peacefully coexist,
assuming no bugs in the software listening to the multicast
advertisements (which Henning mentioned above - a legitimate concern
to be sure, but not the big one).
So yes, the problem only exists if you are running VRRP and CARP on
the same subnet (say, a pair of routers speaking VRRP and a pair of
firewalls backing each other with CARP and pfsync, which is actually
fairly common) and happen to have a host identifier conflict. In a
completely random world the likelihood of this is 1 in 256.
Unfortunately, human nature and a plethora of examples on the
intarwebs makes "interface GigabitEthernet 0/3 // vrrp 1 ip blah"
reasonably likely. The same human nature causes the out of the box
configuration on many products, e.g. pfSense, to be "ifconfig carp0
vhid 1". Presto - outage for everyone on the subnet.
Soapbox time. There are some people who decide that they will only
run FOSS software because of how they feel about software patents. In
my case, I will not run CARP because of how I feel about folks
deliberately violating the interoperability maxim ("be conservative in
what you send and liberal in what you accept") and causing *me* to be
the collateral damage. I think we all have enough on our plates
dealing with legitimate software bugs without having rogue protocols
deliberately interfering with our networks because of a grudge. Is
CARP malware? A trojan? I'm not sure I'd go that far, but it seems
to meet some of the definitions.
Nothing personal Henning (and I like what you did with OpenBGPd and
OpenNTPd) but you'd gain a lot of respect in my eyes, as well as a
bunch of other people's, if you publicly admitted the CARP OUI
decision was a huge mistake. If your lawyers have advised you not to
apologize because of liability concerns (despite that "no warranty"
bit in the BSD license) it's OK - I completely understand.
Drive Slow,
Paul
{% endraw %}