Check out this article in The Atlantic: The Coming Software Apocalypse.
It’s a must-read, although I’m not convinced there is a solution here. We keep pushing the boundaries of complexity because one of our society’s fundamental values is not just innovation and iteration but proliferation. We seem to expect software quality to be maintained by a Darwinian survival of the fittest. That said, how would we regulate software, and who would do it? How do we decide, “this software is good for our society, and this software isn’t”?
The article doesn’t attempt to address these ethical questions. Instead, it focuses on the practical, discussing a movement to step away from code that advocates for allowing the developer to work more directly with ideas and requirements — opening the world of development to a wider range of people, such as subject matter experts with a direct understanding of the problems they are trying to solve (i.e. doctors creating medical systems, lawyers creating legal systems, etc).
As it turns out, the platform that I use to create applications, FileMaker, has been part of this movement, and I’ve enjoyed working with it for precisely this reason. Along the way, I’ve seen it wrestle over time with the same contradictions, becoming more code-like and less transparent. Likewise, the applications we’re creating with it have become more complex — or are often used for middleware in connection with traditionally coded systems — with the result that requirements become harder to trace. Nonetheless, it’s worth acknowledging as part of the movement towards more effortless development tools.
I agree with the article that the next generation of these tools will be fundamentally important. However, I’m pessimistic that they will be sufficient to solve the fundamental problem of humans designing systems that are vastly more complex than we can manage. For all of our sakes, I hope that I’m wrong, and I strongly support any effort to address this issue.