On the need for structure in a budding Enterprise environment…

I am not an “organized” person. I like to go with the flow, be spontaneous, and shoot from the hip. I fully acknowledge that is not always the best way to be in a profession such as computing, but it is what it is. In my professional capacity, I am a proponent of organization on a much larger scale than I could ever do in my personal life. I think Christy is to blame for that. 🙂

At any rate, I started my new job in May, and was thrust into a position where there are 6 datacenter-ish locations (5 DC’s and HQ has a server room with a fair number of live boxes)… still a far cry from the 3,000 Linux boxes we had at Fidelity. In fact I’m certain we don’t have 3,000 servers total here. Maybe half of that, plus or minus.

Fidelity was a large corporate environment, and had evolved as such: there were procedures to operate, procedures to engineer, procedures to plan, procedures to add or modify procedures. It was annoyingly bureaucratic at times, but for the most part, everything worked well. The loudest complaints were over minor crumbs, not failures of the system.

Cue the move to the Wild West of Enterprise. The 5 datacenters are all acquisitions from other acquisitions, and there is no degree of interoperability. I and my cohort that was hired in at the same time, have been tasked with making sense of this mess, but we differ on policy. A lot.

I am the guy that says, “we need to fix x, y, z…and q,r,s,t, and w,v, and b,c,d,e, and f. Y’know, we’d be better off to start with a clean slate and move over what actually works.”  My coworker, would rather fix things by implementing whatever his previous employer used, because it “just worked for them”, but in reality it’s only because that’s what he’s familiar with.

My philosophy here is to use a well known and supported product that can manage and deploy 95% of what we need, and then use supplements to supply whatever else we need. Total cost spelled out in contract terms, and you can make a budget that fits without the danger of surprises. Plan for the least common denominator, and build the environment that a monkey and 2 trainees could run, and any sysadmin in the world who knows anything about Linux can jump in when it’s time for us to move on.

His philosophy is this is free, and that is free, and that is free….we can cobble these things together and make it work. And between these 20 products, none of which actually work together, we can manage everything….theoretically.

Truth be told, if this were *my* network, his method is *exactly* what I would employ. Because I *want* to have to figure things out…because it’s fun. However, when my job is on the line, I want stable, damn the education benefits. I don’t want to have to learn how to diagnose an interface from a monitoring tool and how to parse the output of the logs to be read by a supplementary program that is needed to translate between the app layer and the web layer of the network switches different vlans only to satisfy an arbitrary request from a client that could have been solved in 30 seconds if we were Redhat/Cisco end-to-end. Or even SuSE/Juniper, or CentOS with a Spacewalk server. Or even actually running a solid LDAP implementation. It’s called Penny Wise, Pound Foolish. If it were my own network/server, the learning is a valuable part of the project, not just the cost. In business, the uptime and costs associated with not having downtime is paramount, not the learning.

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *