Magnitude Information Engine

The Magnitude Information Engine is based on what used to be Kalido, which was originally developed by Royal Dutch Shell as a means of speeding up the creation of its data warehouses. This capability was subsequently spun off as the independent Kalido, which was then acquired by Magnitude Software. It provides a model-driven data warehouse automation platform where that model is a business model dealing with business entities and relationships and which captures business requirements. From this, the software will generate and optimise data warehousing deployment for Oracle, SQL Server or Teradata, along with all relevant documentation. It supports both normalised and de-normalised structures but does not support Data Vault, although the Information Engine actually uses a very similar structure that roughly map to the Data Vault concepts). The company is in discussions with Snowflake to support that company’s environment.

The Magnitude Information Engine product can be deployed in the Cloud (Azure, Amazon, or Teradata Cloud preferred) or on-premises and requires a Windows desktop client.

Customer Quotes

“Kalido gives us fast access to vital, global product master data, ensuring maximum efficiency in R&D.”
Shell Lubricants

“Without Magnitude (Kalido), it would be extremely difficult to speed up our monthly reporting, let alone keep pace with business changes and new reporting requirements. Our staff would likely still be tied down consolidating spreadsheets.”
Scottish Power

The key component of the Magnitude Information Engine is the Business Modeller. A partial screenshot is shown in Figure 1. The first point to note is that this has been designed for business users and there is no requirement for either a logical or conceptual model, though you can import models from ERWin or PowerDesigner if you wish. While we will not discuss this model in detail, it is worth mentioning that black lines represent requirements (a client must belong to an age group), may or may not have a credit rating, and the recursive relationship with credit rating indicates support for ragged hierarchies. Multi-value and bi-directional relationships are also supported.

The architecture of the product is illustrated in Figure 2 and a significant feature is that the Magnitude Information Engine is commonly used (around 70% of customers) in conjunction with Magnitude’s MDM solution. The two are closely integrated, with hierarchy management, workflow, master data stewardship and governance capabilities provided to ensure data is organised and validated for reporting purposes.

Figure 2 – Architecture of the Magnitude Information Engine

The way that it actually works is that transactional data such as sales figures is stored in a conventional star (or other) schema but contextual information such as product hierarchies, organisational structures and so on are stored separately. It is this separation that gives the product its ability to manage changes in the business context without affecting the data warehouse structure. Further, change management is not limited to individual data warehouses but applies also to federated environments and to the propagation of changes to data marts. This approach also results in flexibility in the way that business structures are supported. In particular, multiple structures: historic, current and planned, can be supported concurrently, which means that, to use the company’s own parlance, you can keep a true “corporate memory”.

Creating data warehouses is a time consuming and costly business, so automating as much of that process as possible, is a good idea. In addition, it has traditionally been the case that the principal reason for the failure of, or delay to, data warehousing projects has been in assuring good quality and trustworthy data. It therefore makes a lot of sense to have data warehouse automation integrated with master data management though, of course, you can still implement either of Magnitude’s relevant offerings on its own. With specific reference to data warehouse automation we wrote the following when we reviewed what was then Kalido back in 2001: “where the product is unique is on its emphasis that everything will change. You will acquire new subsidiaries, merge with someone else, add new products with new product attributes, add new sales structures, change your reporting methods and generally adapt to meet new business demands. Traditional data warehouses are not well able to cope with such change. Kalido, on the other hand, has been specifically designed to manage changes just like these.” We quote this because it remains valid today. Conventionally, when a business model changes, for example when your organisation takes over another company, or when there is internal restructuring, then the data warehouse has to be re-organised, a process that is often time consuming and complex and which can take many months. Using a tool such as Magnitude Information Engine enables changes to the business model without impacting on the data warehouse and, in practice, it enables the redeployment of a new version of the data warehouse within what may only be a matter of weeks.

The Bottom Line

There are two major issues that Magnitude data warehouse automation addresses. The first is rapid implementation, and the second is support for change. Both of these are important and should result in significant time and cost savings, as well as increased productivity and time to value.