Management / The American View: Dirty Rotten Sysadmins

The American View: Dirty Rotten Sysadmins

The first IT person that a small business hires is crucial to the company’s future. Business Reporter's resident U.S. 'blogger Keil Hubert explores the reason why so many single-person IT ‘shops’ tend to be horrible – and what can be done about it.

If you work in IT or in InfoSec, then you need to accept and embrace the principle that your people are the most important part of your organization. All of the shiny kit in your data centre doesn’t mean a damned thing if you don’t have good people making things happen. Put another way: if you’re a notional ‘head of IT’ (like a CIO, CTO, CISO, etc.), and you’re not leading technical people, then you’re not a real ‘head of IT.’ An IT leader with fully-outsourced services and no technical people is little more than a contract representative liaising with external service providers. That’s a job that an intern could do. You’re superfluous. IT leaders lead IT people, full stop.

You may have noticed that I’ve been in a foul temper lately about personnel issues over the last several weeks here on American View. Lately, I’ve had my fill of the industry buzz about ‘cloud’-this and whatever-‘as-a-service.’ It’s given me the impression that business leaders have lost sight of the crucial, existential truth of IT: if you don’t have IT people, then you don’t have an IT function. Taking that idea to its next logical step: If you don’t have an organic (that is, an in-house) IT function, then your business has no control over its destiny when things go sideways. You’re just a consumer of some other entity’s services, utterly dependent on others’ goodwill and engineering prowess. If that’s how you want to live your life, so be it. Good luck. Just understand that giving dominion over your core IT functions to outsiders means that those outsiders – not you – decide whether or not you get to remain in business.

I understand the allure of outsourcing. I do. I’m no hypocrite; when it comes to my personal consultancy, I outsource my own IT plant because it’s the most cost-effective way to keep my kit running. Since I’m the only technical person in my business, and since I’m obligated to other activities 90% of the time, it doesn’t make financial or operational sense for me to babysit my own Exchange server when I can pay a hosting provider to keep that sorted for me for near-as-makes-no-difference £10/month. I get the appeal of outsourcing. When you’re running a small business – especially a growing business that needs concentrate on your core product or service in order to build your brand – then outsourcing your IT plant makes a lot of sense.

Side view of mechanic in white uniform lying down and fixing under car. Auto repair serviceIt’s the same logic that dictates why I don’t fix my own car. The money I save by changing my own oil is less than the money I’d make by spending the exact same amount of time and effort to write a new book. 

That said, I have no empathy for businesses that are large enough to afford their own in-house IT staff and then choose not to. Once you’ve grown large enough to afford a dedicated management tier, then you’re also big enough to afford dedicated IT staff. If I had enough revenue coming in from my books to afford a full-time office manager, then I’d have me a full-time head of IT as well.

Having said that, I’ve noticed over the year that a lot of small business owners tend to agree with me. I’ve met a lot of people who actually delayed hiring their manager tier in order to get a full-time IT jack-of-all-trades on the payroll. They get it! They seem to have grasped the importance of the role and needed no prompting to protect their endeavours’ futures. Everything’s great, right? Eh … no. That’s where things get strange. I discovered (to my horror) that many of these small business IT hires were utter lunatics; poseurs that did their employer more harm than good.

My initial working hypothesis for why this happened was that non-technical entrepreneurs had absolutely no idea how to evaluate, hire, or lead IT folks. I noticed that non-technical people treated us IT folks like we were wizards: obtuse, domineering, and capable of disintegrating mere mortals with bursts of arcane hellfire. While that analogy is great for the ego, it’s doesn’t hold up in reality. Pro tip, business owners: there hasn’t been a sysadmin born (yet) that can cast fireball in real life. So, maybe it was just a hiring problem.

My second hypothesis focused more on environment than skill: small businesses couldn’t afford to pay market rates for technical talent. A good sysadmin who can run servers, storage, and networks while supporting users, PCs, printers, and phones isn’t going to be cheap. A well-rounded tech can make a good salary with benefits pretty much anywhere. So, if Larry’s Taco Stand wasn’t willing or able to offer the same pay and benefits packet as International Tractor down the street, then all of the good workers will work in tractor manufacturing. Simple supply and demand.

Farmer in tractor preparing farmland with seedbed for the next yearYou can actually make a darned fine living performing IT support for tractors. Modern farm equipment is as technologically complex (and as expensive) as a bloody stealth fighter.

It makes sense that IT workers competing for lower-end jobs aren’t enterprise-grade material. There’s nothing wrong with that; everyone has to start somewhere. Hell, taking a gig in a small business helps a good IT aspirant to flesh out his or skills and to ‘level up’ to corporate standards. This is a good thing … in fact, it should have made my job easier. Instead of fixing an IT person’s mistakes, I could mentor them and build up their skills so that they learned from their mistakes.

Those two hypotheses worked, but they didn’t account for the more egregious errors that I ran into. I noticed that small businesses tended to only employ one technical person which opened them up to exploitation and abuse. I knew from running several termination actions that a charlatan can’t survive very long in the corporate world. The more co-workers and bosses there are evaluating a poseur’s work, the more likely it is that they’ll be exposed and shown the door. But in an isolated, tech-blind shop … a scammer can dig himself in and create a comfy little empire for quite a long while. If there’s no one around that can identify the fraud’s failures, then all of his errors, mistakes, and omissions can be blamed on the ‘evil gods of technology’ with the boss none the wiser.

Evidence supported that premise. Bad hires deliver poor performance. I’ve invoiced far too many business owners over the years for services that – if I’m honest – I should never have had to perform. I’ve provided technical support for companies that had their own dedicated techs who simply couldn’t be bothered to learn new operating systems. I’ve installed networking gear for companies whose IT support staff wouldn’t touch anything but PCs. I’ve rebuilt servers that had been set up by business owners who possessed zero tech skills, but who were too intimidated by their own IT staff person to insist that certain work get done. I spent around ten years politely and cheerfully cashing cheques from businesses who shouldn’t ever have needed my services at all given what they were already paying for in-house IT services.

In many of the jobs that I took, I found that the client’s ‘IT bloke’ [1] was a hot mess of flamboyant arrogance, middling skill, and a grifter’s habit of obfuscating everything technical in a veneer of unnecessary mysticism. These were IT people (in name only) that had grossly over-stated their qualifications during the interview. They just barely kept their employers’ operations running, and didn’t dare let anyone else into their operation that might reveal to the boss that they were (near as makes no difference) clever frauds. The best analogue I’ve found in pop culture were Steve Martin’s and Michael Caine’s characters from Dirty Rotten Scoundrels; con artists that preyed on rich single women on the French Riviera. Frauds without conscience.

robber steals money of the nightstand while woman sleeps deeplyA con artist is willing to tell you whatever it takes to get close to your money, from ‘I love you, darling’ to ‘Yes, I’m proficient with that type of LINUX.'

That’s how I came to the conclusion that small businesses are intrinsically vulnerable to being taken advantage of by unscrupulous operators. Again, I’m not arguing that every small business sysadmin is a villain. Not even most. Rather, I’m arguing that the built-in protections that medium-sized and larger companies have to screen out underqualified applicants simple don’t exists in most small businesses because of the inevitable Catch-22 inherent in all IT departments: if you don’t already have an IT department, then it’s nearly impossible to build an IT department. Put another way, you have to already have a team of qualified experts on staff before you can hire your first qualified expert. The metaphorical deck is unfairly stacked against the small business owner.

The end result was a wicker combination of nearly-insurmountable obstacles: a small business owner with a limited budget and no IT knowledge posted a gig for a single do-everything tech. Being unable to afford top talent, most qualified people didn’t apply. Of the applicants considered, the business owner almost always selected the weasel that seemed too good to be true (that it, was lying through his teeth). The unscrupulous weasel then messed everything up, leaving the boss none the wiser, until the entire IT stack went up in smoke. Inevitably, the boss called a consultant like me in after the weasel quit (knowing that he was about to be exposed). Same story, time and again.

Once the theory fell into place, I realized that it doesn’t have to be that way. I know it seems like it there’s nothing that can be done, but that’s not true. Small businesses may be intrinsically vulnerable to this sort of exploitation, but it’s a solvable problem. This is one of those conundrums that requires some literal ‘out of the box’ thinking … and by ‘box,’ I mean ‘company org chart.’

The trick to this is to realize that you don’t have to go it alone. Just because you’re a small business owner doesn’t mean that you have to do all of your own selection and hiring. Since your objective is to hire the expert that will create, expand, and manage a high-quality IT support function, the best way to find the right fit for your operation is to borrow someone who has already done – or, better yet, is already doing – exactly that sort of work. This is the other kind of outsourcing: instead of buying data services from an external provider, rent human expertise instead.

Corporate business team and manager in a meeting, close upWe’re leaping forward in time, in a sense, to take advantage of the sort of IT head that you’ll have running things once you’re a thriving medium- or large-sized company. 

Specifically, leverage your network of friends, neighbours, relatives, etc. Ask the people that you know and trust how strong they feel their companies’ IT departments are. Do they perceive their IT leaders as technically robust? Honest? Forthright? Are they people that they’d feel confident recommending as a boss for their close friends or their own children? Use those leads to work out where the best IT operations in town are, and then lean on your network for an introduction.

It’s a small investment of time. Spend an hour on the phone or over a drink with a potential partner [2] and be candid about what you’re after: ‘My business has grown enough that I’m ready to hire a person to stand up our in-house IT team. The thing is, I’m not technical enough myself to know how to tell strong talent from drek, and I don’t want to make a disastrous hire. One that might sink my company. So, I need help in screening, interviewing, and selecting a new dedicated IT person. You clearly know how to do this, and I’d like to get your help.’

If you think this sort of request will be met with stony silence or condescending dismissal, you really don’t know tech people. Our kind of folks love solving problems, especially when it entails being positively recognized for your knowledge and performance. It’s a huge ego stroke. It’s irresistible.

Try it and see. Odds are, you can contract a staffing company to get you a stack of strong CVs. Once you have a dozen or so candidates ready for review, ask your new IT buddy to take a Saturday, a holiday, or even a paid personal day to go through the stack with you. Don’t just outsource it: get them to teach you how to read the resumes so that you learn how to interpret the keywords.

Businesspeople in a meetingThe kind of people that know how to read a technical résumé and explain why the author is completely full of it.

Then, once the two of you have narrowed the stack down, pay your borrowed IT buddy a fair day’s pay to lead a few interviews beside you. Let the IT person worry about evaluating technical qualifications while you explain your business and needs. By the end of the process, you can be reasonably sure that the person that you select will be the best possible of all of the candidates that you had apply.

Hiring your first dedicated IT parson is a huge investment in your company’s future. Think of this borrowed talent as a crucial strategic investment. The first IT hire is going to set the tone for every new tech person that joins the operation after that. You want the best possible person running the show from the start, so that all subsequent hires are the best possible team members. The last thing in the world you want is to have your time and money wasted on a lying weasel. Therefore, don’t rely on a ‘gut feeling’ to decide your company’s future, since a clever lying weasel can take advantage of your technical ignorance. Instead, get some outside help and guarantee that your early hire won’t doom the company … and that you won’t have to hire an outside consultant like me to come in and clean up ‘your’ IT guy’s messes.

Speaking of, I don’t do tech support anymore – at all – because of all those disastrous clean-up jobs that I did in years past. I will, however, cheerfully help you staff your IT operation. So will a lot of other senior IT people just like me. You’ll be surprised how eager we are to help a stranger. All you have to do is ask.


[1] It was always a bloke. Every single time. Bad statistical sampling? Or something inherently misogynistic in industry? I have my suspicious, but I can’t prove them. So …

[2] You’re likely already investing this sort of time reinforcing relationships with your suppliers and partners. Think of an allied IT expert as another sort of service provider.

Title Allusion: Dale Launer, Stanley Shapiro, and Paul Henning, Dirty Rotten Scoundrels (1988 film)


POC is Keil Hubert, keil.hubert@gmail.com

Follow him on Twitter at @keilhubert.

You can buy his books on IT leadershipIT interviewinghorrible bosses and understanding workplace culture at the Amazon Kindle Store.

Keil-Hubert-featuredKeil Hubert is a retired U.S. Air Force ‘Cyberspace Operations’ officer, with over ten years of military command experience. He currently consults on business, security and technology issues in Texas. He’s built dot-com start-ups for KPMG Consulting, created an in-house consulting practice for Yahoo!, and helped to launch four small businesses (including his own).

Keil’s experience creating and leading IT teams in the defense, healthcare, media, government and non-profit sectors has afforded him an eclectic perspective on the integration of business needs, technical services and creative employee development… This serves him well as Business Technology’s resident U.S. blogger.

Shares