Too Much Stupid Software

Listen to pod cast

Having spent the last 25 years programming everything from IBM mainframes, Apple II, OS/2, Macintosh, Unix, and Windows systems, using assembler, C, C++, Java, JScript and C# I’ve often thought that what we are doing as programmers is absolutely crazy.  Deep down I say to myself, “there has to be a better way”!   Ryan Dahl expresses this sentiment perfectly on his blog.

We (programmers) are all drowning in complexity and users can tell.  They get frustrated and yell at the screen every time the software does something stupid.  We programmers hate having to deliver imperfect software.  But perfection just seems unattainable on any sort of reasonable schedule which is frustrating and demoralizing.

Some blame management for not giving us enough time to finish the job.  Some blame the tools and languages we are using.  Some blame the layer upon layer of stuff we are building on top of.  Some say we just haven’t invested enough in testing our software.

Either way, I claim it is a complexity problem.  We as an industry have hit a complexity ceiling and we can’t seem to break through.  The biggest, best, most well equipped team in the world cannot handle more than about 50,000,000 lines of code.  At this point they hit a complexity limit that can only be solved by removing stuff.

Everyone I’ve talked to about this acknowledges there is a problem.  But what amazes me is the lack of desire to do something about it.  The usual answer I hear is “let software evolve” and “it’s too expensive to re-engineer from the ground up”.  Well, we’ve let it evolve for 40 years and look at the mess we are in!  Saying it’s too expensive to make a quantum leap into something better is a ridiculous argument because we don’t know what that thing is yet, so how can we possibly know it’s too expensive?

Let me make a bold statement.  Evolution isn’t working. We are actually just going around in circles re-inventing the same old crap.  There’s a radical, scary thought.   So we chase the next new thing, jump on it, hoping it will change the world.  Some get more popular than others, and in this highly connected world, the rate of new things appearing, and the rate of adoption of those new things seems to be increasing.  But in the end we all come away with that sinking feeling that Ryan talks about so eloquently.

They say 50% of the solution is knowing what the problem is.  Well I claim the core problem is that we are using 2 dimensional tools and languages to solve an N-dimensional problem.  Languages and their accompanying tools are really great at two things, functional decomposition of a problem, and the linear execution & debugging of the resulting code.  Well it’s better than nothing, but today programmers also have to worry about threading, security, compatibility, reliability, performance, load balancing, fault tolerance, testability, installation, updating, patching, and so on.  I claim there are about 30 dimensions like this dissecting our code turning it into a big stinky pile of dung, and we are the dung beetles scurrying around busily trying to make the pile of crap less stinky.    So next time your manager asks how you are doing you can say “I’m as busy as a dung beetleJ

It is clearly crazy that we are still programming for-loops in this day and age.  Some say the solution to all this stop writing code, let the machine do it based on a Model Driven Architecture.  The idea being computers will be better at solving these N-dimensional problems and coming up with bug free code that implements the desired behavior.  Sounds good, but I haven’t really seen it achieved yet.

Some say the answer is intentional programming; the problem is we just haven’t been formal enough about all our intentions in code, and once we add that rigor and provide tools that know how to reduce those intensions to code then we’ll be good.  The zed specification language, program proving and all that stuff fit in this camp.

Some say the answer is dynamic languages and/or aspect oriented approaches.  This camp essentially says, screw it, systems should be more dynamic, let me just add my piece of crap wherever I want to add it, and get out of my way.  Interesting approach for cowboys.

Some say we have enough software, we should just stop making more because we’ll probably only make things worse.  We should be happy with what we’ve got. This is the old fart camp and they’re really full of it.

I say we need a quantum leap in computer science, and we need the new generation of programmers coming out of college to grab hold of this idea and run with it.  Don’t be happy with the status quo.  Continually push the boundaries on every front.  Question deeply everything you’ve been told.  Whoever discovers the quantum leap and smashes the 50m lines of code ceiling will rule the software world for the next 100 years and lead us into the Star Treckian realm of billions of lines of code working seamlessly together to create a better world.

If you are still reading you’re probably wondering what I’m smokin’ but I claim this will happen sooner than you may think, and I also have a hunch it may well be triggered by a discovery in the rapidly advancing world of biochemistry.   Life systems are many orders of magnitude more complicated than any software system we have yet built, yet somehow all that complexity is boiled down to some seemingly simple DNA encodings (I say seemingly because I don’t think we really know yet how DNA actually works).   But what is the true DNA of software?  What are the proteins, the amino acids, the true patterns that govern entire complex software systems.  The amazingly cool thing for anyone entering the world of computer science is that we simply don’t know yet.  It is still the wild west.

So I’m a total optimist.  While the state of the art is a big stinky pile of crap, the future will bring new and exciting breakthroughs.  The opportunity for this sort of discovery is reserved to a very small group of folks who are actually looking for it, so it is truly exciting times ahead for them.

Too Much Stupid Software.mp3 (5.14 mb)