What will the next generation of development look like?

A long term associate and friend who has worked the business of assessing software productivity told me not long ago that his data showed that the productivity in 75% of large organizations have not shown significant improvement over the last 10-12 years. Of the remaining organizations, about 13% had improved their productivity, but the other 12% had actually declined.

As I have written elsewhere, the reasons why the software industry has stagnated, if it has, has to have something to do with the complexity of the current software development environment. The software that we develop today is, at one level, much more sophisticated. It is object-oriented distributed, web-based, distributed, etc. etc. The user interfaces are much more sophisticated. We have video objects, Flash objects, Google Map mashups, etc. The data stores are growing exponentially, with unstructured data (email, attachments, multimedia) growing fastest of all.

The the price of all this complexity is the many of our primany software activities have become much more complex to develop than they were just a few years ago. A decade ago, you hardly ever heard the word "deploy". Today, every product has to be deployed over larger and larger universes. Clearly, as we are seeing everywhere, software is becoming more complex at the price of cost, reliability and auditability.

In the next couple of blog entries, I'm going to talk about what we need to do to change our development paradigm.