I keep giving Linux second, third, fourth chances... hoping that one of these days I'll deploy it on something without disappointment or drama. Aside from installing Ubuntu on random laptops with low expectations (run a browser and Wireshark), it just doesn't work out that way.

The latest saga of misfortune came when some colleagues wanted to use Fedora 20 so as to take advantage of some new features in disk cacheing. Something I discovered after some thrashing around is that there is a network manager program which one is supposed to use to make all changes to network interface configuration. It is possible by editing files in /etc/sysconfig/network-scripts and rebooting to break the network, but not configure the network. Between that and attributes that never actually got applied to the interfaces as part of normal startup (LACP hashing algorithm, for the curious), we collectively decided to execute Plan B, which was Red Hat Enterprise Linux 7.0, which at this writing went GA a couple of weeks ago.

So started the real fun. Where's my ifconfig(8)? arp(8)? netstat(8)? route(8)? Hmmm. Not there at all. How'd the network get configured then? Digging around in the system startup scripts I found the answer - confirmed by some colleagues - everything had been rolled into a command called ip(8). Great.

Let's take a brief walk down memory lane, shall we? How long has ifconfig been around, and how common is it? The answer is it's on pretty much every Unix variant with a networking stack and was around in 4.2 BSD (August 1983) and may have been in 4.1c in 1981. I don't want to stand in the way of progress, but having a commonly used utility vanish without so much as a shell script to print out an appropriate error message just seems sloppy. Red Hat folks, let me show you how folks who care about their user/admin community handle the situation:

{% raw %} # uname -a SunOS t3.seastrom.com 5.11 joyent_20130919T215407Z i86pc i386 i86pc Solaris # apt-get install xemacs The 'apt-get' command is not available on this system. If you are trying to install a package, please refer to 'pkgin' that can work with the Joyent binary package repository. #

Pretty snazzy eh? Almost as if they knew that people would likely not read the release notes, and even if they did muscle memory would have them type stuff without thinking. Maybe, just maybe, the customer deserves something better than "command not found" when they type a common command that's been part of Unix-like operating systems for longer than you've known what a computer is, and maybe longer than you've been alive? How about leaving a "tombstone" when you kill something off?

Anyway, I digress. We got things partially working under RedHat 7 after a fair amount of difficulty. I suspect we were getting hosed by some maintained state that persisted across reboots in the NIC card (blade chassis, didn't power things all the way down). In the old days I would have dismissed this out of hand, but after recent experiences with NICs that have everything short of their own OS running on them, I am paranoid. We still haven't managed to get things all the way working, but we're on a tight timeline so I suggested FreeBSD might be a good direction to look in, particularly since there are some good arguments in favor of using FreeBSD for the application at hand, the same way that a decade ago if one wanted to run Oracle the winning choice was clearly Solaris on SPARC.

When I suggested that, I got the distinct impression that people thought that would be Too Hard, perhaps from a configuration management perspective, maybe from a hiring-people perspective. Nobody came right out and whined about it, but it wasn't a warm reception by any stretch.

About ten years ago I ran into a similar situation and a similar response from my staff. I responded by listing all of the Unix variants and Linux distros that I'd sysadminned over the years. Like many networking graybeards I used to be a sysadmin (even took care of a little non-Unix stuff periodically like a VAX running VMS). These days, I do my best to disclaim any knowledge of system administration lest I get sucked in to supporting something which would be a distraction from my Real Job. Anyway, I digress. Once upon a time, there was a notion that sysadmins ought to be able to jump in and figure shit out. Sure it'll take you a little longer if you're not familiar with the platform, and it always took a little fiddling to get your hands around startup scripts and what you had to do to get code to compile (this was long before the days of dependency-resolving binary and source package managers!), but the mood was definitely "git-r-done". These days, it seems that you get the hairy eyeball if you dare to suggest something that goes against the "we're a fill-in-the-blank shop" party line.

To make a point I went looking for the list and came up empty-handed. So I took a swag at reconstituting it from scratch. The criteria to be included on this list was "getting paid to sysadmin it, making it do something non-trivial" or "part of a volunteer sysadmin team for a community resource". Things that don't count include playing with eval units or experimentation, stuff that I booted once to accomplish a particular goal, such as Voyage Linux to reflash the BIOS on an ALIX, or any appliancey stuff like Astlinux, dban, or FreeNAS.

I put this together in the course of about twenty minutes so I'm sure I've forgotten at least one or two things. Anyway, in no particular order, here's the list.

SunOS 2.0-4.1.1 (sun2, sun3, sun4, sun4c) Solaris 2.51-11 (various SPARC, Intel) OpenSolaris/OpenIndiana/SmartOS BSD 2.10 (PDP-11) System 7 (Dual 83) Ultrix-32 (VAX - late 80s) NetBSD (since just after 386bsd days) OpenBSD (multiple, since 2.0) FreeBSD (multiple, since 1.1.x or 2.0? '94 et seq) RHEL 4/5/6 RHLEE 6.2E, 7 Slackware (mid 90s) Debian (multiple, circa 2000-2004) Ubuntu (since '07?) SuSE ('09-'11) MacOS X (10.1 et seq) NextStep (circa 1991) Irix (4, 5, 6) DG/UX (circa 1991-1993) OSx (Pyramid, circa 1991) Interactive Unix (circa 1991, pre-Sun) SCO Xenix (circa 1988) AIX (1990)

Not included are platforms I had some short stints doing development on, but not primary sysadmin responsibilities for: NonStop/UX (Tandem Integrity), HPUX, Altos Xenix, and eval of 386bsd/Jolix or other stuff that I don't remember right now.

So in summary, it's not that I'm a Mac-using BSD-installing hater, but rather a right-tool-for-the-job person who sees applications running on boxes and just wants an OS underneath that can be properly configured to do what we want it to do. I figure that any sysadmin who's any good oughta be able to step up and make it happen. My lack of religious zeal for any particular platform causes me to repeatedly stub my toe on Linux, which in turn makes me grumpy... but I suppose that's the price to be paid for breadth of experience and knowing firsthand that there's a Better Way. Hmmm, maybe the advice I've been giving to the youngsters to look farther afield than knowing how to drive a single Linux variant (poorly) wasn't such a great idea after all, hmmm? :-P