Sorry for starting this with a long quote but it’s important:

“Centuries ago, if you had wanted to build a large structure such as a bridge or a cathedral you would have engaged a master builder. He would have had some knowledge of what it takes to give a structure strength and stability with the least possible expense and effort. He would not have been able to express much of this knowledge in the language of mathematics and physics, as we can today. Instead, he relied mainly on a complex collection of intuitions, habits and rules of thumb, which he had learned from his apprentice-master and then perhaps amended through guesswork and long experience. Even so, these intuitions, habits and rules of thumb were in effect theories, explicit and inexplicit, and they contained real knowledge of the subjects we nowadays call engineering and architecture. It was for the knowledge in those theories that you would have hired him, pitifully inaccurate though it was compared with what we have today, and of very narrow applicability. When admiring centuries-old structures, people often forget that we see only the surviving ones. The overwhelming majority of structures built in medieval and earlier times have collapsed long ago, often soon after they were built. That was especially so for innovative structures. It was taken for granted that innovation risked catastrophe, and builders seldom deviated much from designs and techniques that had been validated by long tradition. Nowadays, in contrast, it is quite rare for any structure – even one that is unlike anything that has ever been built before – to fail because of faulty design. Anything that an ancient master builder could have built, his modern colleagues can build better and with far less human effort. They can also build structures which he could hardly have dreamt of, such as skyscrapers and space stations. They can use materials which he had never heard of, such as fiberglass or reinforced concrete, and which he could hardly have used even if he could somehow have been given them, for he had only a scanty and inaccurate understanding of how materials work. Progress to our current state of knowledge was not achieved by accumulating more theories of the same kind as the master builder knew.”

― David Deutsch, The Fabric of Reality: Towards a Theory of Everything

I have been building since I have awareness with one of my earliest memories, probably around three years old, being the building of a blue Lego seaplane. My Dad brought it to me from a trip and I remember the happiness and the sense of urgency in building it. It must have been the model 371-3:

In same vein, I was reflecting yesterday that as a child the most compelling Disney character to me was Gyro Gearloose (or as I knew him: Proka Pronalazač). That lightbulb lighting up… that’s one of my peak experiences to this day.

Over the years, I have become a successful software builder but there is no explicit theory behind it. So in Deutsch’s sense, and that sense only, I call myself a master builder: over almost 40 years of programming, I have built intuitions and rules of thumb but I don’t think I have an overall theory of software development. Interestingly, I also think that most of the programming intuitions (as in: creating algorithms) I built very early on and that I have built more engineering (as in: selection and prioritization of work limited by resources) intuitions and product (as in: selection and prioritization of work with the purpose of selling the end result) intuitions.

But lately I have been thinking that relying and even embracing the fact that there is no theory might actually be the right thing to do. Software system might be something akin to economic systems: each individual part is well understood but put together, beyond a certain scale, the systems are not entirely predictable. Other systems that have these properties: anything with a dynamic component and where signals take stochastic time to propagate, like traffic and factory workflows. If my intuition here is correct, software engineering can learn much more from say traffic engineering than from structural engineering (which Deutsch uses in his example). As a side note, I also think that building/operating SaaS is much closer to say running a factory with all its logistical challenges than to say building/selling “shrinkwrap” software of old.

In the future I want to consider two things coming out of this:

• Deutsch’s “rules of thumb” and if those can be subsumed with theories.
• Should I really be embracing the lack of overarching theories of software development or is that just my personal preference of maybe even limitation? Do I think this because I have felt that way about “planned” societal systems from very early age? In other words: do I think this because it is true or I think it because it’s comfortable for me?

Last modified on 2025-01-29