The Chaos Factory by Adam Wasserman

I don’t often review books here, but The Chaos Factory by Adam Wasserman attracted my attention. He asks the reasonable questions: “Have you ever wondered why the website of your bank or phone company is so broken and unusable? Do you find yourself asking how this can be? What is wrong with these people? Why can’t it just work?”. That last is a particularly good question when firms like Google and eBay seem to be able to change their web interfaces a lot faster than many traditional companies can – and when many companies are aspiring to be “Mutable Businesses” in a constant state of change and evolution. But, as an aside, remember that Google and eBay just have to work on average, work “well enough”, and just refund your money when they get it wrong; while a regulated financial services company can’t afford to be quite so cavalier about the odd failure.

Adam answers his questions by means of a wide-ranging survey of the history of software development, from the early days of machine code (0s and 1s, often expressed as hexadecimal numbers) up to the current promise of the “industrialisation of IT” – he is a partner at Neonto, which is in in the game of industrialising IT for faster development of mobile and web apps. Adam’s book, however, isn’t really about marketing Neonto (which is a Finnish “high productivity development” tool, written by Pauli Olavi Ojala).

Adam describes early programmers and explains that they weren’t coding Neanderthals (not that Neanderthals were stupid) but really clever people, who coded intricate, elegant, optimised programs in machine code using clever tricks – such as using an instruction in hexadecimal as the value of a constant, thus saving the space and resources used in managing separate constants. Very clever, but it also explains why “clever programming” was a term of disapprobation in my day as a programmer: such programs are very hard to read, almost impossible to change and maintain (sometimes, even if you have the original author available) and broke easily if hardware behaviours changed.

So, we developed compilers and higher level languages such as FORTRAN and COBOL, which people who spoke natural languages like English could make sense of (FORTRAN is “formula translation, for mathematicians; COBOL is “common business-oriented language”, for business people). These addressed many of the problems with the use of machine language, through abstraction away from the “bare metal” (hardware), but brought problems of their own.

Computing evolved towards ever higher abstraction (although sometimes, I’d suggest, it even went backwards, with the introduction of C and early implementations of Windows device drivers, for example), but with each innovation Adam explains why it was introduced – and describes the unexpected consequences that led to its being replaced by another innovation. At each stage he sees both the positives and the negatives – which is a useful antidote to the sort of techno-freak who tells you that everything before him/her was rubbish, and that their new silver bullet is perfect.

For instance, I greatly love CASE tools (Computer-Aided Systems Engineering) but they haven’t had the success I think they deserved. Adam points out that this is partly because they often targeted the wrong problem, automating the clever value-add coding that a good human programmer can do more effectively than a computer, rather than the mundane integration and validation code that just has to work reliably and transparently.

OK, so far so good. The Chaos Factory is an interesting and readable story, if you like that sort of thing, and reasonably accessible to anyone who has met a computer that is automating business. I like the book because it makes sense to me at a technology level. I like it that Adam quotes my favourite guru Fred Brooks, author of the Mythical Man Month (a book about managing computer systems development that is over 50 years old and still relevant today). I like it even more because it adopts modern design thinking and presents the idea of “compiling” a single technology-neutral description of an app design to run on whatever physical hardware you like, be it IOS or Android. That last idea is not exactly new (executable, technology-neutral, unified modelling language models are already a thing) but the trick is in how well and accessibly it is executed, and it might just just solve most of IT’s problems – if an organisation can change its culture to support it.

But I am not writing this primarily for IT and I particularly like the fact that Adam makes comparisons between traditional manufacturing and possible IT industrialisation; and provides a set of “motivations for change” at the end of his book directed at some if the more important stakeholders in business automation: the CFO; CEO; the CIO or CTO; as well as at programmers.

I think the Chaos Factory could be considered essential reading for business analysts, “citizen developers” and the like, who think they can develop useful apps better than IT can. Perhaps they can indeed – but they’ll need to pick up some of the disciplines adopted by the IT group if they don’t want to build apps which work well enough today but become unsupportable, unmaintainable, unreliable “concrete overshoes” for the business over time. I can remember the spreadsheet revolution of the early 1990s and how the bank I worked at (in IT) was soon in hock to obsolescent spreadsheets written in previous versions of the tool by business users, without input from IT – because no one knew how they worked or what would happen if a macro behaviour changed in a new version of the spreadsheet product. Nevertheless, everyone was sure that these spreadsheets, whatever they did in detail, were fundamental to the operation of the bank.

What I am saying, in essentials, is that modern “citizen developers” (or whatever) need to work with, not against IT; and that they must avoid reinventing the bad ideas and bad practices IT has already tried. The Chaos Factory gives people a background in how IT got where it is and why it behaves the way it does; and provides a blueprint for future change, based on this foundation.

This makes it an excellent basis, in my opinion, for going forward with design thinking and the new “Industrialisation of IT” and for reducing the silo walls between IT and the business, using the new business-oriented, high productivity business automation platforms that are becoming more popular now.