On consistency, silos and catalogues

In a previous life, before I became an analyst, I was a software developer. Fair warning: I am about to talk about programming, or at least software development. Please bear with me; I have a point. If you also have a background in development, you can probably skip the next paragraph.

In the same way that every writer has their own style of prose, every programmer has a style of writing code. The differences are perhaps less obvious, at least to the untrained eye, but they are there, as any programmer or developer will tell you. Moreover, style has a great effect on whether or not the code reads well. This often seems a strange thing to say, but to someone who knows the (programming) language, well written code can be read almost like English. This makes it much easier to understand, to debug, to extend, and so on. In other words, it makes the entire development process easier. So, style is important. But unfortunately, good coding style is in many ways similar to good writing: it’s subjective and difficult to define; when you’re used to reading good style, bad style sticks out like a sore thumb; and it’s something you have to learn, either through experience or more often through tutelage. A large component of the latter is the enormous number of books and online resources devoted to teaching good coding style. The sheer multitude of books available confirms my own suspicions, based on anecdotal evidence, that although many programmers agree on the basics, virtually all of them disagree of the particulars of what good style is or, perhaps more importantly, what the best style is. Since I used to be a programmer, let me tell you my opinion, which I owe in large part to one of the aforementioned books: the best style is the one that your team agrees on. Consistency, above all else, is paramount. Even if you have to write in a subpar style, a consistent style, no matter how poor, is better than an inconsistent one. I should also say that, among software professionals, this is not an unpopular opinion.

This brings me to my point. Philip Howard, a colleague of mine at Bloor, recently published a blog about the likelihood and danger of data catalogues becoming restricted to silos, with each department having their own, isolated, data catalogue (you can read about this here). If this happens, a huge chunk of the proposed benefits of those catalogues go out the window. The ideal solution is for each department’s upper management to get together and decide on a single data catalogue to use universally. Frankly, this solution seems obvious, but I’m not sure that many companies are actually going to doing it! Consider the situation: each department wants – or indeed has – a data catalogue, and each has different requirements and different preferences. Each has come up with its own best solution. Sound familiar yet? Just as software developers have collectively learnt to compromise their style for the good of their team, siloed departments must learn to compromise for the good of the organisation. Remember: consistency is paramount, and one good solution in unison is better than any number of perfect solutions in isolation.

If the worst comes to the worst, there are potential technical solutions. In fact, in his article, Philip highlights – and dismisses – several suggestions before settling on knowledge graphs as the most promising approach. However, even the best fixes will amount to just that: fixes. Moreover, they will be fixes to a problem that doesn’t need to exist in the first place. It’s already common to have silos for data, or for BI and analytics. Organisations can and should learn, either from their own past or from their counterparts in software development. Will they? All things considered, probably not.