Applications in highly regulated industries - Mainframe 3.0, Hybrid Cloud and being able to sleep at night

This paper is based on an analyst discussion with IBM, which claims to have a global view of, and special expertise with, “clients that are turning the corner from the early stages of cloud adoption to a much deeper, business-driven mastery of hybrid cloud” – what I’d call hybrid cloud maturity, driven by metrics and business outcomes. Although I’ll use IBM use cases, I’ll try to keep the discussion generic, although we will be assuming very large implementations of highly regulated solutions, and some of these will be on Mainframe v3.0 – the mainframe as just another highly resilient, highly capable server. 

The journey to hybrid cloud is basically a modernisation journey. Don’t throw away something that works (or is certified to satisfy regulatory requirements) if it can be used as part of a modern hybrid cloud solution through, say, an API, and perhaps with a little improvement to the user interface. 

IBM makes the point that highly regulated industries face specific challenges, balancing innovation, compliance and security; but I’d suggest that everything is, in effect, highly regulated these days – even if not everyone has noticed yet – because, say, anti-money laundering and privacy just two governance issues, there are others) are universal concerns.  

What is hybrid cloud? There are probably as many definitions as vendors, but I think it refers to a heterogeneous technology platform (with computing, storage, and services), comprising on-premises infrastructure; private cloud services; public cloud services (not just from one provider, Amazon Web Services (AWS) or Microsoft Azure); and with free orchestration across it all (using cloud APIs). Key characteristics of hybrid cloud include: 

• Build applications once, deploy them everywhere (not only on one vendor’s cloud); 
• Manage applications in one place, host them anywhere; 
• Share skills across the whole environment; 
• Innovate anywhere, regardless of vendor, platform etc. 

IBM points to five challenges that must be addressed, if hybrid cloud is to deliver effective business outcomes. My take on these is: 

1. Complexity

Cloud-based chaos is not an improvement on monolithic on-premises platforms. This is addressed with a clear cloud-services-based architectural vision and enforcement of a cloud API-based implementation. 

2. Siloisation

You need transparent communication over all areas of the hybrid cloud platform. Silo walls are not just matters of technology or vendor jargon but can be matters of genuine cultural difference – for instance, cloud-native cultures delight in risk and value innovation above all else, whereas a mainframe 3 culture focuses on business continuity and reliable service delivery. Most of these cultures will have value, the focus should be on integration and communication, with managed risk. 

3. Security

A rich hybrid cloud environment can can increase the security attack surface, compared to a monolithic on-premises solution. Properly managed, it need not be more risky, but it will certainly be differently risky, so risk and threat analysis must be revisited. before and during the move to a hybrid cloud platform. You will need a unified security program that recognises business objectives, optimises the use of resources, and a culture that places security first. 

4. Financial

Managing resources in a fixed, monolithic, on-premises environment with limited vendors is very different to managing virtualised resources across a multiplicity of vendors in a hybrid cloud environment. You need to be able to manage and optimise virtualised hybrid cloud resources (including those still running on premises) from one place. 

5. Partner ecosystem

All of the stakeholders in the business outcomes from a hybrid cloud platform (including external societal outcomes as well as just internal financial outcomes) must come together to define and build mutual success. 

People and management issues 

Helping clients master hybrid cloud is something any vendor in the cloud space should be doing. Nevertheless, the choice of technology is a client responsibility, and you can’t easily outsource this responsibility to a vendor, who will usually (at best) deliver a working solution on their technology choices, not necessarily the best solution overall for you. Of course, you might decide that the risk associated with a merely satisfactory hybrid cloud solution is less than the risk associated with the time spent finding an optimal solution and delaying modernisation – but this should be your decision, not the vendor’s. And, always remember that “merely satisfactory” will look like heaven from a mismanaged and unfinished search for optimality that requires skills and experience you don’t have. 

A key resource, if you can find one) is an “honest broker”: an organisation (or person) with extensive vendor-neutral (or vendor agnostic) cloud expertise who can advise you on the choices available (you may miss something yourself, that is excellent but poorly marketed) and their consequences; on realistic timescales for modernisation (managing expectations matters); and on “the art of the possible”. 

There are at least two elephants in the room, which you must take seriously in any implementation of hybrid cloud. 

1. Cultural silos 

The people who currently look after the “systems of record” in a major regulated organisation are likely to be conservative and resistant to change – which might put the data that they are custodians of at risk. They are aware of risk and that change might reduce as well as improve service delivery. At the same time, people currently maintaining distributed systems believe in innovation and “fail early, fail small, fail often – and fix it quickly”, they are much more tolerant of risk, which helps make systems Mutable but sometimes failure can’t be tolerated if it means failure of regulated compliance.  

There is a place for both cultures, but you can’t just mix them and hope that they’ll sort themselves out if you buy the right technology. You will have to put resources into merging the two cultures, making sure that risk is managed appropriately and that mutable change is fully supported. Key approaches for achieving this include in-house, no-blame discussion forums and provision of highly skilled mentors with people skills and both business and technology experience. 

2. Single source of truth 

Data don’t have to always reside in one place, and being able to move data to where they can be stored most cost-effectively (e.g. archives on read only media); where they processed most efficiently and speedily (e.g. real time in memory); the edge, where they are available close to where they are used (for low latency); or even to high resiliance, high security “systems of record” data stores (mainframe 3.0); is one of the joys of a hybrid cloud approach.  

It is key to Mutable, where access patterns and business logic structures constantly evolve to adapt to a changing environment. Ideally, an optimising scheduler will run each workload on the most appropriate platform, and this will change (both short term and long term) over time. Data duplication is key to performance and resilience, but you don’t want to get different answers to a query depending on where you go to get the data.  

A hybrid cloud architecture must address the question of “one source of truth” in a robust way. One approach is to store everything physically in one place and to provide remote (cached) access from everywhere, but that isn’t really in the spirit of hybrid cloud, and may have latency and consistency implications – what happens if two apps are changing the same data for use in a subsequent calculation from opposite sides of the world, especially if communications go down? It is better if all data can be logically federated, abstracted from their physical storage locations, which may be anywhere – but that must be built into the architecture from the first.