Costing Cloud Data migrations

One statement that gave me pause when talking to Zoho recently was its belief that it had to run its own datacentres, partly because public cloud is so expensive – if there’s one thing we always hear, it is that AWS etc is cheap.

A moment’s thought at the time told me that for the Zoho business model, public cloud probably would be impossibly expensive. Now here’s confirmation that public cloud can be expensive in practice, from The Register reporting a NASA audit.

An interesting story, I thought – and the analysis of what went wrong is instructive. Currently, NASA operates Distributed Active Archive Centers (DAACs) and is moving it all to the Cloud (AWS). Unfortunately, the transfer of petabytes of data into the Cloud won’t be instantaneous, so for some (considerable?) time, NASA could expect to pay for its DAACs as well as for Cloud storage.

But it is worse than that, and costs are much higher than NASA anticipated, according to El Reg’s reporting of the audit: “Specifically, the agency faces the possibility of substantial cost increases for data egress from the cloud,” the Inspector General’s Office wrote, explaining that today NASA doesn’t incur extra costs when users access data from its DAACs. “However, when end users download data from Earthdata Cloud, the agency, not the user, will be charged every time data is egressed. That means ESDIS [Earth Science Data and Information System] wearing cloud egress costs. Ultimately, ESDIS will be responsible for both cloud costs, including egress charges, and the costs to operate the 12 DAACS.”

I asked Paul Bevan, Bloor’s Research Director: IT Infrastructure what he thought of this issue.

“The Cloud costs issue is generating a lot of interest at the moment”, he replied, “it comes in two parts: “can I assess the likely costs given the nature of the apps I am putting into the cloud”; and “can I or am I monitoring the usage of cloud instances (people tend to spin up servers and provision storage and leave them running even though they aren’t being used)”.

The first question used to be addressed by consultants in fairly costly cloud migration projects assessing what the likely costs would be,” he continued, “but now, we are seeing ArchOps tool vendors using the huge amounts of app and infrastructure performance data now available, to predict usage and therefore costs. Virtana (formerly Virtual Instruments) has a tool called Cloud Wisdom that does exactly that. Such vendors are now addressing the orchestration and management of cloud infrastructures more generally, in order to monitor usage/costs more closely.”

On the specific issue of Cloud data egress cost, Paul points out that OVHCloud, the French Cloud company, claims to be the largest European Cloud provider and does not have any ingress or egress charges. It also has a global footprint of data centres. Type OVH into the search bar on the Bloor home page and you will see the four articles he’s written about OVHCloud.

So, there are tools that NASA could have used to cost its Cloud migration better and also the possibility of avoiding egress costs altogether. That is worrying for the American taxpayer, but I wonder how many other organisations aren’t costing Cloud migrations properly? There will always be applications that simply aren’t right for the Cloud or not right yet, and there are now more tools and more examples to guide users new to the cloud before they jump in and make an expensive mistake. One would think that people would model costs for moving to the cloud as carefully as they would cost out a new datacentre, but perhaps not always. And in the NASA case here, the old joke business model: “FREE ENTRY! exit $99.99” – may not be so funny after all.