It is a truism that most businesses fail. Specific, clear, unambiguous statistics to back this perception up are hard to come by. The blurring of the meanings of ‘merger’, ‘acquisition’, ‘buyout’ and ‘failure’ have made those statistics even harder to interpret. Still, I think it is a fair generalization. Certainly, as I look back over my career, 90% of the companies I’ve worked for in the various branches of high-tech are either out of business, or so transformed (perhaps by acquisition) as to be unrecognizable. I’m not quite vain enough to think it’s because of me.
So why do businesses fail? A google search will yield lots reasons people offer up with good intent. Look them over, it’s an instructive exercise. As you do so, ask yourself how much the lack of systematic focus on stakeholder requirements and the corresponding emphasis on accumulation of domain expertise is actually a secret, semi-conscious theme running through many of the reasons given. Also notice how seldom it is mentioned explicitly and clearly.
Most start-ups think they do all right when it comes to accumulating domain knowledge, or, more narrowly, when it comes to framing requirements. They can’t be bothered to add detail that isn’t needed right now to get the next milestone in on time. Companies that outsource learn the hard way that they need to work harder at it than they expect, or they do not survive.
Yes, most companies, most engineering teams think they’re doing fine, until, deep in the midst of their heroic labors, they are shocked awake by the realization that small oversights, wasted efforts, and little pieces of time lost to rework mean that the critical business opportunity has been missed. Numbed, they try to pick up the pieces (or watch as others do) in the aftermath.
It’s hard to escape conclusion that this is the result of a systematic shortcoming in our approach, that we’re suffering from some blindness born more of the habit of looking elsewhere than of any fundamental defect in our sight. What could be the source?
In the midst of our Promethean efforts to bring the next great technological miracle into the world, we forget that in the Greek myth the hero-titan who suffered to win fire for mankind had a twin brother from whom he was inseparable. Epimetheus’ strength was“hindsight”, his curse forgetfulness.
Software development is hard precisely because software’s utility approaches so close to the threshold of conscious, and our brains are designed (have evolved) to accommodate the details of tool use automatically, with practice, in our goal-oriented absorption in the task at hand.
What we need is the balance of a little (just enough) Epimethean perspective building to anchor our goal-driven hurry.
I’m suggesting that it has particularly to do with the process of requirements gathering, because by its nature it is something we have to do with deliberation, even though we are inclined (both organizationally and individually) to want to push the task ‘into the background’ of our awareness as soon as possible in the name of ‘productivity’.
We find it painful slowing down our process of understanding, stifling the impulse to think in terms of solutions (and often jumping prematurely at the first one) rather than ‘living in the question’ long enough to know we’re heading in the right direction.
<aside>
I don’t mean to go off into a long digression into mythology, philosophy or, God forbid, post-modern deconstructionism here, but I will pause to point out that I’m not by any means the first to draw this connection. The philosopher Bernard Stiegler was at great pains to point out that we systematically forget Epimetheus and his role in our world, with tragic results. At one point the authors of the Wikipedia article summarizing his work Technics and Time paraphrase Steigler as observing that “a constitutive blindness, forgetting and idiocy accompany the fact of being technical”. This isn’t an indictment of nerds. The intended use of the term ‘technical’ is broad enough to encompass most human tool use, and by extension, most human culture of any sort. It is intended as a statement about the make-up of our conciousness.
</aside>
I’m saying that we’re capable of being better at this, but that it will take systematic, organizational, collective effort and iteration after iteration after iteration. Unless that effort is part of our ongoing way of doing engineering, of doing business, we don’t have a ghost of a chance to master the situation.
We need to pause in the here and now, between our Promethean questing and our Epimethean regrets, to systematically grow and cultivate our awareness of the present, of what we know and of how to communicate that.
October 3, 2009 at 10:21
This articles resonates well with a lot of articles I published on project failure. This article, why projects fail, lists the #5 cause as “Poor Project Planning”.
This paragraph: “Yes, most companies, most engineering teams think they’re doing fine, until, deep in the midst of their heroic labors, they are shocked awake by the realization that small oversights, wasted efforts, and little pieces of time lost to rework mean that the critical business opportunity has been missed.” Is essentially about bad requirements gathering.
IMO, gather the requirements well, prepare (but don’t spend a lot of time preparing, or the project will never get started), and start the project, and most probably you’ll do fine.
October 3, 2009 at 11:00
Thank you for your response.
I agree that Requirements gathering is a crucial part of project planning. The reality is that it is impossible to bring it to completion during an initial “planning” phase and be done with it.
Teams informally seek out supplementary information and, in practice (if not in name) continue the requirements gathering and refinement processes well into implementation. This isn’t a failure of process, it’s the natural evolution of engineering projects.
One thing I’ve tried to emphasize strongly is that the ‘form’ of our requirements expressions and our use of terms and our conscious processes do not reflect this reality well enough yet, and that causes lots of problems downstream.
I would advocate being (in a sense) less formal initially, so that your requirements are structured in such a was to facilitate their refinement and elaboration throughout the project, and carries over from project to project.
October 3, 2009 at 17:32
Your last point the comment is spot-on. Having some fluid requirements (initially) is a very good thing.
November 4, 2009 at 17:53
It will take including women on your teams…! And listening to them
November 4, 2009 at 19:57
I quite agree. Just yesterday I had the conversation I repeat, in various forms, up to several times a year. It usually revolves around one or all these questions:
Why are there so few women in software engineering?
How would things have to be different to attract more of them?
How would things be different if more were participating?