By Martin Brampton, 6 July 2004 09:35
COMMENT Contrary to popular belief, software development is not a purely technical or analytical task. Martin Brampton looks at what this means for outsourcing, recruitment and the future of IT.
It is sometimes thought that writing software is a technical matter that requires analytical skills. There is an element of truth in this, but it is by no means the end of the story. Building quality software involves creativity quite as much as analysis.
In fact, the modern world tends to exaggerate the role of analysis. Perhaps this is because of the central place we give to science as a way to understand human experience. We are inclined to think that science is the best example of clear thinking and a rational approach to questions.
In a way, this is true, for as the western world has over the last few centuries placed less emphasis on authority and more on reasoned criticism, so we seem to have made great strides. Yet the view that science simply discovers natural laws has often been criticised, both by philosophers and by thoughtful scientists.
Physicists have generally led the way. When Newton discovered simple mathematical rules that appeared to explain all the complexities of the astronomical world, people could be forgiven for thinking that the world was absolutely governed by such rules, if only we could discover them. A few centuries later, studies of phenomena at the atomic level turned all this upside down.
Heisenberg told us that atomic events are intrinsically uncertain and that the more we try to measure accurately, the less sure we can be of the results. It even seemed that the observer had an essential part to play in atomic events, exemplified by the notorious and uncomfortably placed Schrodinger's Cat.
Nowadays, physicists are much more inclined to think they are creating interesting models that give us insight into the world. They have handed over the belief that science tells us all about the essential laws that govern the world to biologists, who frequently take a rather literal approach to scientific theories.
What has this to do with software? Good software actually creates a new aspect to the world; it generates new possibilities. Only the simplest of systems are merely an automation of something that already exists. Indeed, modelling systems on already existing practices is often a quick way to failure.
A simplistic description of the software development process suggests that all the interesting thinking takes place at the design phase. The completed design can then be mechanically turned into code. But this hides some of the most important processes. It is only when the design comes into contact with the unthinking computer that its quality becomes apparent.
There are many different ways to create a computer process that will assist a particular activity. Some are far better than others, in terms of their economy of operation and their ability to give desired results in all circumstances. The problem does not determine the solution, leaving huge scope for the creative talents of software developers.
There are many practical implications to this. One is that outsourcing of software development is not as simple as it is often portrayed. While there are undoubtedly talented developers among the outsourcers, they are necessarily more remote from the environment into which the software is to be deployed. This can severely limit their ability to create a solution that benefits the client organisation.
Another is that recruitment practices often place too much stress on skills such as knowledge of programming languages, and too little on creativity, innovation and communication. Software developers should have the talents to contribute to the development of an organisation, rather than being regarded merely as mechanics applying a technical skill.
Yet another is that we should recognise that IT is unlikely to have reached a terminus. The great fascination of software is that it has constantly revealed new and unexpected possibilities. There is no reason to suppose that this is at an end. Thank goodness.

Comments
There are 16 comments. Join the discussion
1. anonymous
I think you mean Schr?ger's cat
[Ed. note: You are correct. The article has been changed to reflect this.]
2. Sean
Programming is neither an art nor a science. Just like hardware, software configures components to produce a design that gives reliable results.
Poor design, poor methodology and inadequate testing leads to buggy results that waste everybody's time.
An Art!, don't make me laugh!
3. Mark SPLINTER
I love you Martin, please run for Prime Minister.
4. Rick Kline
Devil's Advocate? hmmm I can see the point if you compare, say a tool maker and all his skills to a software engineer. Where a tool maker uses his skills as a "artisan" to make something, and with experience the tool maker can dream up an idea put it on paper and fabricate a one off tool. As can the software engineer.
The danger here is in calling a software engineer an artist. We all know that software projects start out with a defined objectives to reach the end product. In my industry (games dev.) software projects never go to plan, and are held together by senior developers and projects managers.
The "art" in software development is to actually deliver a product. Not making it.
Geezee I can see it now... all our software engineers running around with "arty farty", "toys out of the pram", "BBC-esque" lovey dovey attitudes... Screaming I can not work in these conditions... my mouse mat clashes with my T-shirt...
5. Dick Wall
Software development seems to me to have a lot in common with crafts. Typically it depends upon on the job learning and benefits from good relevant experience. The skills often seem to be more readily caught than taught.
Similar to a musician or a chess player for instance. And like musicians and chess players some of the good ones are good communicators and some are dreadful.
Of course theoretical understanding is worthwhile training and will help produce rounded professionals but is over emphasised in many evaluations IMHO.
Interesting view of the history of Newton's cultural effects. For 17th century minds and even young ones today it is pleasantly surprising how rule governed things appear to be. Chaos theorey has also reduced our belief in the ordained ness of the universe. Arguably the solar system is for instance chaotic, mathematical models require simplifications and assumptions that are of unknown reliability.
6. Alfred Reading
Schrodinger not Heisenberg was the authority on the cat problem.
[Ed. note: You are correct. The article has been changed to reflect this.]
7. Jeremy J. S. B. Hall
I agree completely with the premise that part of programming is a creative art. This is especially true for the software that I develop - computer business simulations for management development and business training - the management training equivalent of the flight simulator used to train pilots.
Although in polite society (talking to my customers) I emphasise the seriousness of what I do (developing simulations). At social events I describe myself as a "corporate cartoonist" - a person who develops mathematical cartoons! For computer simulations it is not possible to use traditional (heavy weight) software design approaches. As you develop the software you see different ways of providing learning functionality and revisit the software (very much along the lines of the agile, XP programming approach).
Often, like many creative people in other disciplines, I solve programming problems awakening in the middle of the night - as happened last night (although it is quite possible that this was helped by an amount of red wine).
I feel that the major joy of programming is the creativity.
8. David Quinn
I have been in the IT business for over 30 years and Martin Brampton is absolutely spot on. The HR errors are exemplified by their penchant for exclusively hiring people with IT degrees, when as Martin says you need a mix of skills and personalities to develop computer systems. Also outsourcing rules out the absolutely essential ties of trust and commitment which fuelled IT projects and can never happen adequately across contractual boundaries. IT is actually phenomenally hard at many different levels. These practices are now making it nearly impossible, which is why all those public sector systems fail (publicly) and many more private sector systems fail (privately). It's a no brainer so it is tempting to conclude that senior businees management lack brains.
9. Werner H.
Schr?ger's Cat was actually a Red Herring.
10. Dr John H Woods
Off topic: Why the snipe at biologists? Just interested.
11. kevin johnston
OhMyGod: My first post to a bulletin board. Yes, first ever. I was a virgin. (I think technically I still am a virgin until I click to Submit, yes?)
Why advertise that I'm a virgin? What difference does it make? Perhaps I hope to be excused if I make an error, because of my inexperience? That is "If I'm right, it proves I'm really smart; but if I'm wrong, I'm still smart, just inexperienced". Puh-lease. Smart? Unknown. Insecure? Definitely.
Or maybe I'm advertising just because the first time is unique, special, and I want to include as many people as possible in the celebration. ("My first time was a gang thing.") But really, losing your virginity is not such a big deal; I can prove it. (That's going to be a running joke in this essay, but I don't expect you to get it yet, because the race hasn't started yet. I hope in retrospect you will find it amusing, but that's all I can say at this moment.) Now, getting it back, that might be a significant event. But it would probably be premature to be planning the party until I actually lose it to begin with.
So maybe forget that it's my first time. That's not what I came to talk about anyway.
I agree with Martin B's statement "software development is not a purely technical or analytical task". In fact, I can prove it. (Yes, of course I will, what did you think I meant, "I can prove it, trust me"? Puh-lease. But not just yet. Please be patient.) (That was the starting gun, and you can get the joke any time you want now. But don't worry about a slow start, there's still plenty of time to catch up.)
I disagree with Sean's statement "Programming is neither an art nor a science". I can disprove it, he contradicted himself: "... poor methodology ... leads to buggy results". Methodology is science; no science, bad results; ergo, good results requires science. QED.
Sean says A and B are both false, but I've proven that B is true. This proves Sean wrong, but it doesn't prove A true. It is true, I can prove it. (But not just yet. Please be patient.)
(By the way, proving that B is true does not prove Martin wrong. Martin said B is not the complete truth. That doesn't mean B is not true, it just means that B alone is insufficient. It is true that eggs are required to make a souffle; it is not true that with eggs anyone can make a souffle; the eggs alone are not sufficient. Martin actually claims that A and B are both true. Proving B to be true is not inconsistent with Martin's claim, so it doesn't prove him wrong. But neither does it prove him correct. If A is false, then Martin is wrong, even if B is true. But in fact I already said, A is true. If there are no objections, I'll prove it right now.)
Software development is the act of creating software that did not exist before. It's not the same as software copying (creating a duplicate that did not exist before, of existing software), or software moving (introducing software to a location where it did not exist before, although the software itself did exist before in a different location). (And it's not the same as software running, or jumping, or playing tennis either, but I won't treat each of these in detail; the proofs are available in any good dictionary.)
More precisely, it's the process of creating *correct* software that did not exist before. Given a sufficiently accurate definition of correctness, software can be written and compared with the definition. This is the science part: Measure, compare, accept or reject. The problem is, at the outset the definition doesn't exist either. A definition must be created that did not exist before; in fact, more precisely, a *correct* definition of correctness must be created that did not exist before.
I see that I've come all the way around a loop, and I don't intend to go around again. Don't get me wrong, I enjoyed the scenery, and the exercise felt good, and really, isn't it a beautiful day? (I can't prove that it's a beautiful day, you just have to decide whether to
12. Rico Smythe
Spot on. How many times have I head that Web-development skills are trivial (usually from associate field - like 'IT support', 'Project Management' etc).
Its amazing how many people know how to script, but er haven't the time themselves :-)
13. kevin johnston
Oops. Sorry. I didn't know about the length limitation. Here's the continuation. It might overflow again, I'll just have to wait and see:
trust me or not. I wish I could reassure you that I don't mind which way you decide, but I can't prove that either.) Actually, I *may* go around again someday, just not right now. And if *you* want to go 'round again, by all means go right ahead. I'll just wait here.
Creating anything that does not presently exist is a near magical feat. More precisely, creating a concept of a thing which does not exist is the cool part; constructing an embodiment of the concept often can be contracted out. (No, no, I didn't really expect you to be fooled by that; of course you have to construct a First embodiment, to convey the concept of what the contractor contracts to construct. It was just a little joke.)
A concept is difficult to describe. My dictionary defines "concept" as "thought, notion, idea"; but the definitions of each of these is "concept". Rats.
Wow, I just had a thought (whatever that is), and I checked, and it's true: The definition of every single word in my whole dictionary is circular. I can prove it; look:
"aardvark n: circular"
"aback adv: circular"
"abacus n: circular"
(Honest, serious question: Are these two jokes original? ("My dictionary is full of circular definitions" and the definition of every word is the word "circular".) I don't think I've heard either one before, but I'm not absolutely certain.)
Here's my best effort: "A concept is an experiential phenomenon, in which a unique neural excitation state is experienced as representational of a corresponding unique combination of sensory or self stimuli". Egads. As in, if you are stimulated by a rock, the sensory input evokes a unique neural excitation state experienced as representing stimulation by a rock.
But it's the "self stimuli" that make it complicated: *Whenever* this neural excitation state is reproduced, you have an awareness of the experience of being stimulated by a rock, even in the absence of sensory input. The neural state can be self stimulated, and you experience stimulation by rock; without a rock.
You're still reading? I'm honored, truly. Thank you.
So concepts are neural excitation states representing whatever it is you experience when that state occurs. They can represent sensory stimulus, allowing you to experience (or recall the experience of) the sensory stimulus even in the absence of the physical sensory stimulus; but they're not limited to representing the experience of direct physical sensory stimulus. Self stimulated neural states can represent experiences that have never been associated with any sensory stimulus.
I beg my pardon, but what the heck experience do you experience in the presence of a neural state that's never been associated with a physical sensory stimulus? Abstract thought. I guarantee, no one in history has ever had a direct physical sensory stimulus of the neural state representing the experience of considering the meaning of the word "concept". The neural state representing the experience of considering the meaning of the word "concept" can only be excited by self stimulus. (Or by reading this essay.)
Okay, I'm tired, but I'm close to my conclusion. I hope I ... can make ... it.
Creating software requires the prior creation of a definition, which requires the prior creation of a concept, which didn't exist a priori.
If a concept never existed before, then that neural state never existed before. When you experience the experience represented by an original neural state, by definition it's the first time you've ever experienced the experience it represents. How can you know what you're experiencing if you have no experience of ever experiencing it before?
I don't know. That's the part I don't understand.
But I think the novel neural state self stimulates subsequent neural states, and thereby becomes associated with those concepts, exce
14. Rico Smyth
What worries me is that our creative youth aren't painting, making happy tunes, dancing, or writing great fiction.
Instead, the next Rembrandt may be toiling over an insubstantial, ethereal, intangible, fleeting web-site in a room without a view.
And conspiring to hide there creativity (along with everone else), under the complexity of IT.
The sooner we're out of the "monks with quills" stage of web-development the better.
(sorry went off on one).
15. kevin johnston
(I didn't know it was gonna be turned into a mini-series. This is the final episode.)
except that somehow it's a novel combination of the existing concepts, and voila! You have a novel concept. But almost miraculously, you can express the novel concept in terms of distortions of other concepts.
Expressing novel concepts is art. If you express a novel concept of imagery, you have an original painting. Art. If you express a novel concept of sounds, you have original music. Art. If you express a novel concept of characters, events, and plot, you have an original novel (that's *doubly* new!). Art. And if you write software according to a novel definition of correct operation, you get a new computer behavior. And that's art too.
And finally, FINALLY, just one more proof: The sensory experience of losing one's virginity is no big deal, because you can self-stimulate the neural state almost any time you please.
16. anonymous
Dear Editor - article is still not right; it says "Schrodinger" not "Schr?ger" (as the comment pointed out).
We British are careless with our accents, aren't we - how many times do you see in the press: "Bjork" instead of "Bj? (happens to be my niece's name so I am particularly aware of it!)
In some languages, accented characters are actually separate letters of the alphabet with a completely different place in the A-Z order, not just "funny" versions of the unaccented characters.