Saturday, February 25, 2012

Why tech support is the most important job in IT

When I was young I made the impulsive, somewhat stupid decision to drop out of high school and go work for an ISP I had set my mind on getting a job at. As a kid in the early 90s I had a very isolated computing world, and my ISP was a large part of it. They hosted newsgroups, an IRC server linked to a major network, sent detailed, technical announcement messages, had a really advanced website with lots of graphs, and had really smart people who would answer the phone 24/7 to help me get my external USR modem connected via serial. I wanted to meet these wizards who were up at 2:00AM setting up ISDN lines; I kinda wanted to be one.

Thanks to the bohemian owners and leadership at the ISP (and knowing one person who worked there via an extremely small IRC channel), I got a full-time job there one week before I turned 17. In hind-sight, I wouldn't have hired me with whatever color my hair was that month, but there I was on day 1, sitting on the floor of the office and being taught how to crimp CAT5 cable, because they were moving their datacenter around and that's all I did for about 2 weeks.

Over the 2 years I spent there (and 1 additional year at another similar company), I took about a million phone calls, knew how to setup every email client in existence off the top of my head, could configure dial-up modems (and, later, Cisco 675 DSL modem/routers, and even later, PPPoE DSL) while playing nethack, and could run through basic network configurations and troubleshooting without even trying. I had front row seats to the ILEC, CLEC, and DSL broadband revolution that happened. They built a giant datacenter that I got to watch come together every day, and play a minimal role helping them set it up (mostly via physical labor). Towards the end of my arc there, I was getting on the network gear and bringing up new routed blocks for colocate customers. That ISP was the highest concentration of smart, kind people I have seen so far in my life, and it was the best job a nerdy teenager could ever hope for.

And, eventually, I hated it. 3 years of total immersion in open source nerd world and generally doing my job well left me way overqualified (in skill) to do tech support. Unfortunately, I didn't quite have the skills or, probably more importantly, the maturity to make the often difficult jump from customer service to internal support (aka system administration for me), and I got increasingly frustrated with being sent easy but relatively advanced work from admins (like editing zone files) while helping a customer setup Outlook on the phone.

So I made another impulsive decision -- I quit my job, said fuck IT, and went to college. I didn't even make it through high school so I never expected or really wanted to go to college, but I got talked into taking a class at the local community college by a friend and unexpectedly found it very enjoyable. My mantra during college was to a) not take any computer or engineering classes and b) learn stuff I didn't already know.

5 years later, I had my BA in English and felt like a much more developed person. I also didn't really care about computers anymore -- my home LAN used to be quite impressive and was now down to one desktop and one headless Linux box -- but I knew way more about how they worked than just about everyone I met in the philosophy and English circles at my large, public university. Although I did catch one of my English advisers using Pine one day and gave them props (and mentioned mutt).

I made my life as easy as I could using technology while in school, but this was pretty limited. On campus I rarely carried a laptop around and mostly used the lab or library computers to download stuff from my home apache server and print it. In hindsight, I would have loved to have had a Macbook Air and Dropbox, but they didn't exist yet and one wouldn't have been a very practical purchase.

During college I was fortunate to have my writing ego stroked slightly by doing several contract gigs for Pearson Education. They kept asking me to write instructional-type stuff on something I knew stuff about, and I kept delivering it. It was a pretty awesome part time gig during college.

Right out of college I knew getting a job in anything related to English-degree-land would be an uphill battle -- to say the least. I didn't even want to be a writer. In fact, I was sick of writing papers, and although everyone told me I should go to grad school, I never seriously considered it. I thoroughly enjoyed college but my favorite part was graduating and it being over.

I threw my resume at my editors at Pearson and offered to relocate but it wasn't going to happen. So, after a quick month long road trip to drive around the states and see some friends, family, and clear my mind, I resigned to the fact that I would have to dust off my computer skills to pay the bills.

Within a couple months I got a low level job doing QA for a small software company -- which I felt was a good mix of computing and other skills I had picked up, and they slowly realized I could do a lot more than QA so I ended up doing a little bit of everything. But after about a year they weren't doing great and had to 2/3-time me. Shortly after that, I lucked out and landed sort of my dream job at a somewhat dysfunctional company in desperate need of a nerd who was motivated to untangle their mess of Linux servers. By the time I started, they had already fired the last admin. It has been an uphill but productive battle getting their servers in order and out of the stone age, and I've come to really like the place and the people.

What's even better is there is enough technical talent around that the help desk duty is minimal and my desk phone very rarely rings. It took about 6 years and a detour through the world of literature, but I made the leap from support to admin.

For a short time I let the relief of not ever having to setup another fucking Outlook install again sink in. It was a huge blessing and I subconsciously told my brain to start removing that dated information -- time to free up that space for another literary criticism paper.

One evening I was sitting at home (probably watching TV) and my phone rang. It was one of my bosses and I knew he was out of town visiting a big client, so I answered. He tells me he's sitting in the client CEO's house and they need help with his email program -- it's Outlook, and it's not working correctly. I tell him I am way rusty on this kind of thing, but I will try. We aren't even sure what version it is but the phone is passed over. A few minutes later, I've got my laptop out and the correct Outlook screenshots and instructions up and am helping the CEO of a multimillion dollar company setup his personal email. It goes perfectly -- he is an easy person to give tech support to because he happily follows the instructions -- it takes about 10 minutes total, and he is very appreciative, as is my manager, and the CEO of my company, who is also there.

There was no parade when they got home but I immediately realized that, as much as I don't want to admit it, I had to have my IT knowledge include basic tech support. I immediately got on our internal wiki and started writing basic documentation on stuff we setup or need to troubleshoot commonly, created some virtual machines of various versions of Windows and various versions of common software I would probably care about some day, and started refreshing my knowledge of email clients, browser settings, general Windows 7 troubleshooting, OSX troubleshooting, and mobile device setup.

I viewed this process not as going backwards in career, but as reinforcing my fundamental computer knowledge. I thought about it in terms of cars (which I know nothing about): a mechanic that can take apart and rebuild a motor but couldn't check the oil would be fired. I felt like even though my system administration and network engineering skills were becoming more advanced every day, I should still have a solid fundamental computing knowledge. To me, that started in tech support, so that's what I built my foundation on. My tech support knowledge is far from perfect but it's solid enough that Google is only complimenting it (as opposed to guiding it) -- most of the time. And to be honest, it's more about communicating than knowledge.

One job of a system administrator is to do cool shit with beefy hardware, but at the end of the day you and your closest IT peers are going to be the ones most impressed by the fact you migrated from physical machines to an HA cluster of virtual servers -- there is now a lot more magic available and it seems to work okay too!

Another job of a system administrator is to just making the stupid system work. Some days this means using your computing hammer on the database servers at 2:00AM, but some days it means explaining the difference between POP3 and IMAP to a new hire when all of their email seems to have disappeared. These tech support skills are useless if you don't actually use them, so make yourself available (but not too available) and be friendly (but not too friendly).

Make the stupid system work and don't be a jerk in the process -- it's a balance. Until you get to know the person a little bit and they come to trust your computer skills a little bit, treat everyone who needs help like those grandparents who would call in with a skeptical demeanor, have a rocky start, but be so thankful when you finally got it all working.

Tell people what you're doing, explain it to them, don't treat them like idiots -- use that English degree (heh). Most of the time this is totally appropriate and welcomed, but sometimes it isn't. Maybe the next time someone POPs all their email off and begins to panic, one of the people you've explained it to before will take care of it for you and you'll never even hear it. You can teach people to fish without making them crab boat captains.

Although the vast majority of my job involves evoking the dark magics and pleasing the blood lords (like any Unix admin), I still help a lot of people with a lot of basic stuff, they are still thankful when it works, and I still crimp a lot of fucking cables.