The rise of storage flash: do your homework to overcome the confusion

The proliferation of Solid State Disks (SSDs) and flash memory seems only to be matched by a growing choice of formats, capabilities, prices – and confusion. There are also a growing number of flash-only storage start-ups, perhaps spotting a slow spinning disk terminal decline. What is going on?

Initially, SSDs offered relatively small amounts of fast storage for a high price, so it seemed as if flash would be a slow-growth market. Now flash-only companies such as Violin, Whiptail (part of Cisco), Pure, XtremeIO, Nimbus, Solidfire and Karrimario have all emerged – to some extent preceded by Larry Ellison’s Fusion-io. However, each is positioned to address a different niche.

Meanwhile, the larger storage vendors (HP, NetApp, EMC and the like) are injecting flash wherever they can get a performance boost (and no doubt planning for a flash future behind the scenes) as they continue to use spinning disk for the bulk of storage.   

Flash can in theory be installed anywhere a performance boost is needed, either host-based or storage array-based. It is used for caching and processing of data when received and as a straight swap for spinning disk to boost storage read performance. Often, compression and de-duplication of incoming data is now carried out in flash memory; this greatly reduces overall storage capacity needs without noticeably impacting performance, thereby obviating a need for a de-duplication appliance.

However, SSD’s reputation as a spinning disk alternative has suffered from a tail-off from the initial performance boost as increasing numbers of ‘bad blocks’ cause data to be re-directed to built-in spare capacity; random writes are not especially fast…but these things are being addressed.

For instance, ‘high-end’ SSD-only vendor Solidfire now aggregates blocks of data before a physical write, greatly boosting write performance. This is anyway only an issue for constantly updating data; it does not apply, for instance, to read-only applications or to long-term archive where the data is typically written only once. This shows some applications are already becoming better suited to flash.

Solidfire now offers up to 3.4PBs of storage, through 100 nodes in a cluster; this achieves linear scalability and 7.5m IOPS or perhaps 20-50 times typical disk performance. Some of this performance is achievable because it can carry out 16 reads in parallel (with no physical heads on spinning disk constraints). Modularity, using standard x86 server technology, goes with automated load balancing across the nodes; this facilitates incremental upgrades and guaranteeing performance levels which can be set according to quality of service (QoS) requirements. Then add to this high availability and support for multi-tenancy.

Does this sound expensive? Solidfire says its solution works out at about $3 per GB. Most other vendors offer much less capacity, which is fine for smaller organisations, and they may also achieve a lower-cost per GB. We should also remember that flash produces major savings over disk in terms of power usage and heat output, to factor into any purchase and running cost calculation. So we can begin to see why flash might cause a stir in some quarters.

However, the above also tells us that the software and firmware backing up the flash modules will play a big part in overall performance, resilience and appropriate use-cases – and new SSDs with greater capacities and performance levels are appearing almost weekly. (So, for instance, Solidfire’s boosted capacity came through it being able to quickly support the new Samsung 960GB SSD – the largest SSD to date.)

Linear scale-out expansion and multi-tenancy capabilities are especially attractive to cloud providers, while a reliable QoS regime fits well with SLAs. So, it is no surprise that Solidfire CEO Dave Wright told me that his company was best-positioned to target enterprise data centres and public clouds.

For me it is becoming clear that flash storage will soon become a no-brainer in some situations and applications, with spinning disk retreating further towards the lower-performance end of the spectrum.

However, any prospective purchaser should thoroughly check out what each vendor offers and how his solution will be able to respond to technology advances – alongside the organisation’s own needs. It is also still early days with plenty of twists and turns to come. We can surely expect some start-ups to fall by the wayside before the SSD market matures – as it will.