The Problem with IT, Part I
There have been a number of comments on a recent post that have glanced sidelong at a problem domain with which I’m reasonably familiar… and since we have a number of digital gearheads who comment frequently on this blog, I thought I’d use this as an opportunity to start a series of posts about technology.
Not fun technology.
No games or silly gadgets or even really cool evil overlord bases or zombie survival houses – that’s all postworthy material but we’ll keep that to the proper locale. Actual technology that gets stuff done in organizations. Movers. Shakers. Candlestick makers.
Before I get into the nuts and bolts, I’ll offer a bit of a reading list as a first post. I can’t offer an explicit look at the last 15+ years of my working experience as a base to start this conversation, but I can give a list of things that I’ve read over the years that will start to frame a part of the conversation… for those people that may have read them.
This is not exhaustive, it’s just the books that happened to be on the bookshelf closest to my desk. Neither does it happen that I think all of these are of equal value (for example, I disagree with a bunch of Carr’s conclusions because I think his observations are in a limited context). But every once in a while I wonder what other people would offer as recommended reading if I was to take up their job… and since this series was kicked off by Jaybird’s comment: “One of my friends recently asked what classes I took to become a system administrator…”
… here’s a beginner’s reading list for “How to be a Systems Administrator”. If I can ever get my Amazon Affiliateness squared away, links will be added.
Building Stuff
- “The Design of Everyday Things
“, Norman, ISBN 978-0-465-06710-7
- “The Sciences of the Artificial
“, Simon, ISBN 0-262-19374-4
- “Working Minds: A Practitioner’s Guide to Cognitive Task Analysis
“, Crandall, Klein, & Hoffman ISBN 978-0-262-03351-0
- “An Introduction to General Systems Thinking
“, Weinberg, IBSN 0-932633-49-8
- “General Principles of Systems Design
“, Weinberg & Weinberg, ISBN 0-932633-07-2
- “Bringing Design to Software
“, Winograd, IBSN 0-201-85491-0
- “Coders at Work: Reflections on the Craft of Programming
“, Seibel (interviewing Everybody Else), ISBN 978-1-4302-1949-1
Building Stuff (With Other People, for Actual People) or Managing People (to Build Stuff) or Recognizing When Your Boss Is Bad At Their Job
- “Making Things Happen
“, Berkun, ISBN 978-0-596-51771-7
- “The Mythical Man-Month
“, Brooks, ISBN 0-201-83595-9
- “Managing Humans
“, Lopp ISBN 978-1590598443
- “Agile Software Development
“, Cockburn, ISBN 0-201-699699
- “Professional Software Development
“, McConnell, ISBN 978-0321193674
- “Real-life Knowledge Management: Lessons from the Field”, Kazi & Wolf, ISBN 952-5004-72-4
Building Stuff (that Fails Safe) or Mitigating Horror (by Building Stuff Correctly) or Dealing with People (Who Are Panicked)
- “Essence of Decision: Explaining the Cuban Missile Crisis
“, ISBN 0-321-01349-2
- “The Politics of Crisis Management: Public Leadership Under Pressure
“, Boin, Hart, Stern, & Sundelius ISBN 978-0-521-84537-9
- “Managing Crises Before They Happen
, Mitroff & Anagnos”, ISBN 0-8144-0563-0
- “Crisis Management: Planning for the Inevitable
“, Fink, 978-0-595-09079-2
- “The Essentials of Risk Management
“, Crouhy, Galai, & Mark, ISBN 0-07-142966-2
Dealing With Stuff (That Other People Built) or Managing Things (That Other People are Building)
- “Global Information Technology Outsourcing: In Search of Business Advantage
“, Lacity & Willcocks, ISBN 0-471-89959-3
- “Does IT Matter?
“, Carr, ISBN 1-59139-444-9
How To Speak to the Non-IT (Business-Type, Primarily) Folk
- “IT Governance
“, Weill & Ross, ISBN 1-59139-253-5
- “The New CIO Leader
“, Broadbent & Kitzis, ISBN 1-59139-577-1
I’ll do the Amazon links…Report
I’d say (again) “I have to get that fixed some day”, but there’s this guy who keeps fixing that problem for me so until he gets hit by a bus…
… hey, that’s part of Part I.a. Spoiler alert!Report
P.S. – There’s probably no Amazon link for the KM book.Report
Which is weird because the third link when you google it is, like, the Amazon.UK link to it.Report
My copy is creative commons; you used to be able to download it off their web site.Report
My own short reading list:
Writing Effective Use Cases: Cockburn
The Pragmatic Programmer: From Journeyman to Master: Hunt and Thomas
Secrets and Lies: Digital Security in a Networked World.
An observation about Global Information Technology Outsourcing: In Search of Business Advantage . Literally everything in that book is happy talk.
There’s a difference between a remote developer and offshoring. If you can’t get rid of your offshore team, insist on getting those jamokes to check their code into Subversion every night so you can run svn diff to see if they’ve actually been doing anything. And make sure it all compiles.
Here’s how I manage an offshore team. I get them all on Skype and talk to them. I tell them I’m not about to play little cultural games: I want them to give me an ombudsman, one person I can tell anything negative to and he’ll relay it to the hapless recipient. If they’ve got a problem with me, the same path back to me. I tell them there are only two things I’ll ever fire them for: lying to me or the client and saying anything negative about the team to anyone outside the team. India is GMT +5:30, I do meetings on their time.
I spent quite a while specialized to failure, specifically to the failures of offshored projects. Want to outsource? Plan for me or someone like me to stop by when your cut-rate developers fail. Or plan to get fired. Or both. You will not get some offshore coder to grasp the concept of a health care claim when he doesn’t have employer-provided health insurance.Report
I have the Use Cases and the Lies book. Cockburn and Schneier are both a couple of favorites.
Your observation about the global outsourcing book is not quite spot on, but it’s close enough. They mention all the appropriate risks and drawbacks and whatnot, but it’s all buried under the momentum of, “everybody else is doing it, so you will too, so here’s how to do it!” instead of “everybody else is doing it, but you should approach this with Extreme Caution, and here’s why!”Report
The worm has turned on global outsourcing. Tell you a story from 1999, when it was in its heyday. So six failed servlet projects arrived on my client’s doorstep from India. They were so completely outraged, they pulled me off my project and had me re-interview 200 H1-B consultants they had on the premises.
So I set up an interview room. The TCS manager would bring each one in, in turn. Asked the first one. “Explain the methods on HttpServlet” Guy looked at me like a deer in the headlights. I asked him a few more questions, it was obvious the guy knew nothing. But you look at his resume, he’s done everything but write the guidance system for the Apollo command module.
So I’d go on the next guy. He would know the answer to the first question but stall out on subsequent questions. After three of these go-rounds I realized what was happening. The TCS manager was taking the questions out and giving them to the next guy in line, who would feverishly try to find the answers before his turn came. So I sent the manager packing.
It was carnage. I did uncover some rather good programmers, but not many. What I did uncover was the fact that the TCS manager was strong-arming kickbacks from his subordinates. These poor jamokes were crammed into apartments, often six to a bedroom, living in penury. I got quite a few of their H1-B visas cut over to reputable consulting firms.
So finally, after a couple of weeks of this, the client asked me to interview someone from the same consulting firm for whom I was working at the time. I threw my hands up and stormed around and exclaimed how unprofessional it was to interview someone from my own outfit, etc. The client’s IT director grinned at me and said “You’re getting sick of this, arncha?” I said I was, so he put one of the full-timers on that horrid Angel of Death job.Report
The worm has turned on global outsourcing.
My brother in law works for a company right up the outsourcing alley. About 12-15 years ago, they began the process of relocating all their manufacturing to China. Since then, the move definitely increased profits despite all the headaches/increased costs that re-education and shipping delays and lead times and what not entail.
NOW, however, the most recent internal corporate numbers seem to show that having production located in China is not profitable ceteris paribus, but relocating back to the states would require enough costs that leaving procuction overseas is still more cost effective – at least for the time being.Report
These days, it’s almost more cost-effective to retool here in the States and leave the old factory over there, intact.
For almost a decade, roughly the 90s, I helped move factories from Japan to the USA, to the “Right to Work” states. I watched all these problems, only in reverse, patiently advising the Japanese engineers on how to deal with Americans. There were a few things the Japanese really liked about Americans, particularly their willingness to deal with a problem immediately, on their own, getting out the toolbox and tightening up a clamp or coming up with some ingenious tool made out of a coat hanger to deal with a dangerous corner of the robot.
I’m watching Apple get a black eye with all the abuses at Foxconn. See, for my money, the worst aspects of outsourcing are encapsulated in the local management. They’ll grin and promise they’ll run a humane operation. They never do. They put their family members in, incompetents all, they extort kickbacks from the workers, they’ll put any sort of lie on the status reports, they’ll cut corners, they’ll tolerate all sorts of abusive behaviours, the QC is a bad joke.
If you’re going to set up shop overseas, the first hires are always some old man and his wife who’ve lived in that town forever, educated people, hopefully school teachers if you can get them, respected people. They’re your bullshit detectors, your eyes and ears, they know everyone. Never think you understand that situation overseas. You’re always going to be an outsider. Manage your own payroll. Don’t pay bribes. Run a full-fledged intelligence operation: it ain’t your town and don’t ever let your guard down and relax. Confident, cocky, lazy, dead. That’s the order of these things.Report
Hey, you guys are like jumpin’ ahead to post 12.Report
Almost. wait till the price of gas goes up a bit more…
Knew a guy who was speculating in tools, of all things. Every factory that shut down, he was there to bid. Found a locked toolchest once, right about to be sent out to the scrapheap…turns out it had Edison’s notebooks in there. Those went to the museum (for safekeepings. don’t keep fragiles in warehouses)Report
I also forbid the use of PowerPoint or Visio on any of my projects. If you can’t write a Use Case without resorting to Boxes and Arrows, you will never work on one of my teams.Report
Interestingly, I find an odd flip: I know guys who can write a use case just fine, but can’t logically diagram program flow to save their life.
This is atypical in programmer populations, I’m given to understand.Report
Not as odd as you might think. Most competent coders think in code.Report
The problem is, not all good coders can think outside of code.
Really, really good coders can think in multiple programming languages at the same time, and write code that takes the best features of one language while covering over the limitations in others.
Great software developers can do that while simultaneously talking to users and bizfolk, all in the same meeting.Report
My girlfriend is currently learning several programming languages at once. So she had to write a little program to do numerical rounding in Java, without using the Math classes which handle it.
Dunno if you write Java or not, but the teacher handed out some brain dead example code with everything in the main method. I explained how to invoke the constructor of the containing class and she just got Very Cornfuzed. Had to sit down with her and demonstrate how constructors work.
When I solved the problem, I farmed out the rounding work to a separate class, which also confused her greatly. She retreated into the bedroom in angry tears, coming out in a few minutes. Turns out the teacher had never bothered to explain how to think about objects and classes. Precious creature, love of my life, trying to conform to what the teacher wanted and all he’s doing is confusing her. Every Java class has a constructor, whether or not you’ve actually written in the empty constructor.
Though I’ve made scornful noises on the Boxes ‘n Arrows crowd, I started my copy of Dia and put up a UML class diagram. See, I explained, this is how I think about a class. This second row is for members. The third row is for methods. Now look at my Java. Same-same, there are the members up top and all these methods below. Properly named methods look like verbs.
Now look at this other class, all it does is some math work, it doesn’t have any business in with the visual elements collecting the input.
Everything real in the world is an object. The coffee cup sitting on this table is an object. It needs methods, like fill and lift and drink, which might just involve something other than coffee. I could even fill it with sand.
Suddenly, she sat up straight and said “It’s just like the real world.”
I swear, Patrick, it’s easier to turn a competent Liberal Arts grad into a good OO coder. At least they know how to talk to people and sort the world out in practical terms.Report
I’m looking forward to this series of posts. Though I am not an IT person, I’m getting involved in a lot of projects at work as the ‘business user’ for new systems.
I spend half my day talking to consultants or analysts about how my team does their job and where I think all the stuff we use originates – always afraid that if I don’t explain something the right way we’ll end up dropping nine figures on a system that creates more problems than it solves.
Maybe all this will improve my understanding of what they’re really up to.Report
I spend half my day talking to consultants or analysts about how my team does their job and where I think all the stuff we use originates
Wait… they’re asking how your team does its job?! Hold on to these guys.
The popular thing in EMR is to tell doctors how to do their jobs.Report
Excellent point, I actually like the firm we’re employing, it’s not something I’ve had a lot of exposure to (many companies in my field are running billion dollar businesses on Excel and an AS400).Report
That’s because the docs don’t know what will help them!
So you start the iteration: I, the programmer, think you’d like to see allergies (cause that’s easy to code).
You respond: “No, I just want a big flashy warning if I’m going to prescribe something they might be allergic to”
… this is just normal programming. The thing is? Docs got a TON of information, and displaying information conveniently is vital to them getting their job done quickly and efficiently.Report
Wait… they’re asking how your team does its job?! Hold on to these guys.
The popular thing in EMR is to tell doctors how to do their jobs.
This is becoming more common in education, too. Administrators,some of whom have never been in the classroom and some of whom got out of the classroom and into administration as quickly as possible try to tell us front-of-the-classroom schlubs what “best practices” are (mostly derived from the fevered imaginations of Teacher Ed folks, who are not themselves widely renowned for teaching excellence). Almost never does anybody seriously ask, “what do you actually do, and what works and doesn’t?“Report
This is also in post 3.
One of the biggest problems in the IT industry is that IT people assume that they are working for the dude who cuts their checks. That is usually the top dog in the organization for whom the IT guy is working.
This is not the person you are working *for*. This is the person you are contracting *to*. You’re working *for* the end user.Report
Isn’t this generally a problem that starts with the person cutting the checks?Report
Plinko, the fundamental problem of 99.999% of all organization charts is that IT get parked under accounting (because no one else wants it). Accountants live their lives looking in rear-view mirrors, great at the past but horrible at the present and worse for the future.Report
Isn’t this generally a problem that starts with the person cutting the checks?
That is often where it starts. That isn’t where it ends, though. This is post 4.
IT people have a very strong tendency to skew towards a set of behaviors that express themselves as enablers, when it comes to their employer (the guy/gal who signs the checks). This isn’t surprising, as their job is self-selecting for enablers (people who help other people get stuff done are correlated highly with people who help other people get stuff done even when the stuff they need to get done is stuff that is either (a) their own fault or (b) something that doesn’t generalize).Report
IT has historically never regulated itself, as the lawyers and plumbers and electricians and even the goddamn beauticians have done, requiring some semblance of qualification for its membership. IT isn’t treated like a meaningful part of management because it hasn’t behaved professionally. Until it does, it will continue to be treated like so many shipping clerks in the warehouse.Report
Imagine, if you will, the top ten most important IT “regulations” (as you’re using the term) of 1992.
Now imagine the top ten most important IT regulations of 2002.
Now imagine the top ten most important IT regulations of today.
Now strain a bit and theorize about the top ten most important IT regulations of 2022.
Off the top of my head, it seems to me that these top ten lists are either *NOTHING* like each other (maybe #10 one year is number #1 another but absent from the other one) *OR* they’re broad to the point where they’d also apply to lawyers, plumbers, electricians, and the goddamn beauticians (which doesn’t make them useless, of course, merely broad to the point where they only apply to IT because they apply to everybody).
IT hasn’t had a chance to sit still long enough to behave professionally.Report
This is, like, post 8 or something.Report
“You may wonder why every IT department has a couple of fat guys who wear Hawaiian shirts when everyone else in the building is in a suit…”Report
No offense Patrick, but you need some revision control on these posts. 😉Report
This was just a reading list!
You guys showed up before I put up the banquet table and started talking about which wine would go with what potential main courses!
(this is actually a good thing)Report
More meat! 😉
and some syllabub!Report
I hope the next one includes a full table of the planned posts so we only discuss what’s in scope.
Report
I expect IT to operate with an apprentice -> journeyman -> master certificate program, exactly like any other engineering discipline. There are plenty of constants to this business, starting with the fundamentals of security. The current state of professionalism in this discipline is appalling.Report
Most of the folks that I know in this business began on a path of being self-taught. The ones who aren’t are the ones who got hired in the 90’s as part of the boom where companies just wanted someone who could type and were willing to do on-the-job-training from there.
I don’t know of anyone, I don’t think, who got into the business after getting a bachelor’s degree in IT… now, that’s not to say that I don’t know anyone who doesn’t have a bachelor’s in IT, but these folks were folks who got degrees in it because they were doing this sort of thing anyway (and one of them joked about how Chinese Engineering students take Mandarin to fulfill the language requirement of their degree and how it felt like his whole degree was like that).
I think that it would be great to have an apprentice/journeyman/master program… I don’t know that any other engineering discipline has the same percentage of self-taught people that IT has.Report
that’s because it’s easy to self-teach.
back in the old days, they used to teach two or three kids a year, even at the big schools. cull ’em and make them wizards.Report
May God save us from the self-taught. I started out writing line BASIC and little assemblies. Got myself a C compiler and treated it like a macro assembler.
Got my first real job writing C and realized I had to unlearn all my bad habits. I came up under some masters of the craft who taught me to build a test alongside my code and data normalization and the guts of MVC and Patterns.
But the people who influenced me the most were my clients, who taught me to comport myself like a professional, to honor my users, to put data first, to concentrate on how money’s actually made and how not to rush to implementation.Report
I don’t expect everyone to share this viewpoint, but I’ve now got about thirty consulting gigs under my belt now and I’ve seen the inside of more server rooms than is strictly good for anyone.
Here’s the way IT shops historically set up. When the company was a tiny little thing, some kid built their website. He got hired to maintain it. He put in the LAN for their first offices. He never bothered to finish college because he was making pretty good coin and had an office, and nobody could fire him because he was the only guy who knew how to reset the router. The Sales Department wanted to put in some cockamamie system, Accounting another and since he wasn’t a money maker, both departments got their way without any say-so from him. He mungled around and learned to set up a copy of SQL Server to support these incompatible systems and never did learn how to get those systems to talk to each other.
But he was Uncle Dick’s prize pig and though both Sales and Accounting thought he was useless, the worker bees resorted to some de-facto scheme where the company actually ran on Excel spreadsheets, not off their shrink wrapped departmental systems. Periodically, at the end of the month, the harried worker bees would key in the data off the spreadsheets into both systems so they could run the reports. Everyone worked lots of overtime. Of course, neither system was correct and Accounting made a lot of entries in the General Journal.
Then Wonder Boy learned to write some PHP. Thereafter he made a big enough mess to anger both Sales and Accounting. Seems one of the salesmen had a gambling jones and sold his password to a competitor to keep his bookie off his back.
That’s usually where people like me enter the picture.Report
Haha. (rubs hands together) Get a copy of Writing Effective Use Cases and write those use cases yourself. And make those consultants sit in your cube for a few hours a day, attend all your meetings, read all the paperwork. Treat them like new hires.
You’re the subject matter expert. Do not let these guys produce a specification without your buy-in. Don’t let them write gobbledegook: if you don’t understand their spec, it’s wrong.Report
This is all pretty good advice. Also what Will said.Report
Thanks, Blaise, I’ll check that book out.Report
This is post 3.Report
I see Blaise already added one of the essential Schneier tomes, but where’s the DeMarco? Martin? Gang of Four?
It’s more than a little appalling how poorly most large projects are run nearly 50 years after Brooks…Report
I joke about UML. “I use about five percent of UML. But I use it all the time.”
Tom DeMarco’s software management stuff is now kinda old hat. Even he admits as much. Poor James Martin and the CASE fad: CASE, like the Rational toolkits, is a solution in search of a problem, a solution which creates its own set of problems.
The craft of software is maturing. There will always be fads in the universe of craft work of any sort but they tend to orbit around a central core, a core increasingly dominated by patterns, as chip design is now dominated by VLSI. If software development is poorly managed, that’s in no small measure because of poor managers.
Here’s how I manage a software project. I learned this methodology from an old McKinsey guy. I was young and he was old and now I am old and still follow it. I always bring in my projects on time, usually a little early. Decide when you’re going to deliver then divide the project into four quarters.
Quarter 1. Play. Work out which tools you’ll want to use. Get everyone’s development environment up and running. Get buy-in from IT, always do your security model first, use the OS security and the model the corporate security people have set up. Implement your security model and get sign-off from IT, make the developers register with it, gives IT a chance to exercise the proposed security model. Get your database up on a box somewhere, build your database hooks, get the DBAs on it and do not annoy them.
Find your alpha user, get the business analysts talking to her, it always seems to be a her for some reason these days. And not the manager, either! Management thinks in exceptions, users think in rules and it’s rules you’re after. Establish a rapport, convince the alpha user she has the final say-so, make sure management knows she’s on the team. Set up your version control. Exercise it. Get with the IT guys and work out where dev and test and prod will live. Write your Hello World-s, tinker with the security until you’ve got a solid pattern in place. Write and exercise a deployment tool. This will take a fairly long time, so just budget the first quarter to getting this stuff set up properly before you write a single line of application code. It’s always the Three Blind Men and the Elephant in the first quarter.
Set up a Skype channel, maybe several and put everyone on it. Set up a wiki and put everything of substance on it, including status reports. Get management used to looking at it: cuts down on meetings. I don’t let a meeting last more than fifteen minutes. My old man could preach a sermon in 15 and he said if you can’t get it out in fifteen, might as well shut up. Meeting-itis is the bane of my existence: it’s the worst time sink of all.
Quarter 2: Model and Controller. Q1 should have produced the business use cases. Do a first pass over them, derive your data model and the controllers to operate on it. You won’t get it right the first time: plan on it going sideways about three times. Do not succumb to the urge to load “test1, test2…” data, get the DBAs to provide you with real-world data and a lot of it: you might need to write a lot of data transfer code to populate your Model but that’s why you’ve put a whole quarter into it. Write the testing scaffold to exercise the Controller elements as they appear. Build a few Views to get results from the Controller. Instrument everything. Keep the alpha user in the loop, make sure she comprehends what’s being modeled.
If, at the halfway point you realize the spec is changing, and it always will, get the client to sign off on what the Model will support at this point and plan on another development cycle to follow. The client will understand the need for a fixed target to hit: they’ve made promises to senior management. If they get the basics on time, they can get a new budget outlay. The spec must be frozen at the halfway point or you’ll always fail.
Quarter 3: Views. Plan on three full iterations of your View development. By now, your use cases should be complete, your Model in pretty good shape and your Controller is talking to the Model effectively. Here’s where you get the whole user community involved. Do not demo your own software, let the alpha user run the meeting. If the alpha user isn’t confident enough at this point, you haven’t been doing your job. Your acid test is the alpha user’s confidence level.
Quarter 4: The Shakedown Cruise. Deploy to test and get the user community set up to use it. If you’ve done your Model correctly, you can keep it populated with reasonably up-to-date data. Do not train the user community directly, that’s the alpha user’s job and it’s the business analyst’s job to support her. Often little quirky complaints will arise: take them all seriously. Sometimes it’s just as simple as changing the order of a few entry fields or setting up some lookup scheme for customer support or writing some report for Sales. Lots of little stuff.
But using my scheme, if you ruthlessly adhere to the quarter system, you cannot fail. You might deliver something a bit different than the initial spec, but everyone’s been apprised of the changes along the way. If you’ve delivered on time, you will get more work or a job offer. Take the first, politely decline the second. You’re building the aircraft, not sweeping the hangar.Report
Carr is that guy who thinks information makes us dumber, right? Yeah, I don’t agree with that either. And he has a stupid last name.Report
Data != Information
Carr != Stupid (either of you) ;_)Report
Information doesn’t make us dumber, but it makes dumb people think they’re smart, which is arguably a worse problem to have.Report
This is T-shirtable, DD. Great comment.Report
Here’s a current IT story; the kind that would make me grit my teeth even if it wasn’t happening to me.
I make my federal student loan payments online. Recently the Dep’t of Education changed its page for payments. And now I have to re-register, as though I had never had a username and password before. And they don’t bother to explain that on their website.Report
James, you didn’t fall for that phishing scam did you? Oh well, they have to pay for Obamacare someway. 🙂Report
Damn, you’d think I was old enough and wise enough by now.Report