Building out a cloud-based critical app for a bank - SEAM number 8, a developer's meet-up

I’ve just been to a very good developers’ meetup, SEAM #8 at the Metal Box Factory, on Bankside. Well-worth attending, although it places a premium on social skills – I wonder what happens to us old-fashioned programmers, who prefer talking to computers rather than to people?  Although, I do think that people skills are essential to building the right applications for the business, of course, instead of just any old application done right 🙂

It was interesting that I knew one of the presenters, Anthony Kesterton, now at RedHat, but at Rational when I first knew him. We’re still on the same wavelength, I think! IMO, in the rush to adopt new technologies, and new nomenclatures, it is important that we don’t throw away the body of knowledge gained with approaches such as Rational’s. Some things have changed, some things haven’t, and “reinventing the wheel” is neither Agile nor efficient.

The presentation “style” and level at SEAM was very much aimed at potential new adopters, but Anthony tells me that this was requested for this particular event. He also presents at events where audiences comprise of mostly experienced Kubernetes users.

I got the impression that, for this audience, OpenShift/Kubernetes was still pretty cutting-edge – lots of attendees knew of this stuff but I think only one was actually using Kubernetes. Anthony, however, points out that although Openshift v3 (the first version to use Docker container format and the Kubernetes container management software) is fairly new (GA June 2015), Kubernetes etc have been used within Google since well before OpenShift v3 was released.

Apart from the particular technology names used, however, little of this was really new to me (and I do keep up with DevOps etc) – once again, IT is (in part, it seems to me) reinventing old ideas under new names… But with some improvements – I hope – and we may avoid some of the old pitfalls this time around (if we recognise them).

It is worth remembering the existence of RedHat’s Atomic Host, a version of the RHEL (RedHat Enterprise Linux) OS optimised for containers.

Ansible

Ansible claims to be a simple “human-readable” OSS automation language/engine. I’d question “human-readable”, for all kinds of human (perhaps “citizen-developer-readable” is better), but Ansible automation is far easier to read and maintain than shell scripts etc. In the mainframe-oriented world, I think that there are already lots of “intelligent schedulers” equivalent to Ansible Tower (the Red Hat commercialised enterprise framework for controlling, securing and managing Ansible automation with a UI and RESTful API) but it is a bit new in the distributed Linux world, although there is some competition.

The Ansible message is Simple (basic programming skills, tasks execute in order), Powerful (it offers app deployment, configuration management, workflow orchestration) Agentless (it uses OpenSSH and WinRM, which is efficient, and more secure than, possibly exploitable, Agents). 

There’s an overview of how Ansible works, in more detail, on the Web, and also an interesting comparative article, dealing (in brief) with Ansible, its history, relationship to Ansible Tower and its competitors (Chef and Puppet, chiefly).

Gluster

Gluster is “just” software-defined storage, optimised for Cloud, containers and OpenShift (it is “container-native” – and claims to be “nimble file storage for petabyte-scale workloads”). It is OSS, made available on a commercial opensource model by Red Hat. Enterprise-friendly features include media streaming and active archives. It offers mature NFS, SMB and Object (Swift) interfaces. By the way, “Swift” refers to the OpenStack object store component, not the Apple language – well, it confused me.

I think the Gluster highlights are scalability (no metadata server), multiple protocols against the same data, and self-healing high availability features.

Gluster terminology to be aware of:

• Brick: basic unit of storage, “mount point”;
• Subvolume: a brick after being processed by at least one translator;
• Translator: “Most of the functionality of GlusterFS is implemented as translators, including file-based mirroring and replication, file-based striping, file-based load balancing, volume failover, scheduling and disk caching, storage quotas, and volume snapshots with user serviceability” – Wikipedia;
• Volume: logical collection of bricks/subvolumes;
• glusterd: process to manage glusterfsd processes on server;
• glusterfsd: process to manage one specific brick;
• GFID: 128bit file ID in GlusterFS;
• (Trusted) Storage Pool: Group of GlusterFS servers that know and trust each other.

Critical bank application

The critical app for a bank was developed by Accenture (and the presenter, BTW, doesn’t believe that Agile is always the right choice for serious regulated applications, which might be some sort of indirect endorsement of the use of Agile Openshift/Gluster here, when it is).

There was the usual “complex diagram becomes nice simple diagram” slide that supports every “digital transformation” presentation – but it was fairly plausible. The technical architecture here has to cope with 10 million messages a day (using IBM MQ, a very well-understood and well-tuned message queuing product). It uses microservices, fully automated builds (to Docker and OpenShift Container Platform v3.3), runs on Atomic Host OS and provides an “industry standard” OSS automated delivery pipeline (i.e., DevOps). It’s built on a high-availability AWS cloud infrastructure (three availability zones in one Region). Network traffic is over private leased lines rather than the Internet (using Amazon Virtual Private Cloud and AWS Direct Connect).

The deployment pipeline includes automatic code vulnerability scans and static code analysis, which is very welcome. But, this is “necessary, not sufficient”, I think, as you still need to be sure that you are building the right code, as well as code done right – the presenter agreed. The pipeline includes manual triggers (gates) for promotion to next environment and to production.

Challenges noted:

• Gluster FS, it’s new – team needs to get up to speed with its approach to storage fast; integration with OpenShift ecosystem is ongoing; its stability still improving; 
• IBM MQ, it’s being used in a new way, inside containers, so there are few existing use cases to measure against; and there are new levers for tuning performance, not everything is directly under your control;
• Security – is still catching up with container and public cloud approaches; you have to consider security at each layer and make sure that each level works properly with other levels, in security terms.

Bottom line

All-in-all, this SEAM meetup was well-run, socially fun, and informative – an excellent resource for developers working in a fast-changing environment. The venue was also a lesson in the physical security often implemented around IP-generation these days – at one point, I wondered if I was ever going to be able to escape! But at least someone thinks that developers matter.