The first wave
The first instance was the virtualisation of the disk array some 20+ years ago. It was arguably StorageTek that pioneered the concept of an array that completely liberated or abstracted the link between physical and logical storage. Since then storage industry responded with continual improvements to the disk array. Some of these improvements originated in academia – an example is de-duplication of stored data. However, the original concept of storage layer abstraction is as fundamental to smarter storage today as it was then. It would be unimaginable if customers would have had to continue buying pre-virtualisation arrays. Turnover of billions of dollars created a very healthy and profitable segment in the IT industry. Some disk array super-users told us that a feature such as volume cloning could account for as much a third of the value of a smart array.
The current wave of virtualising storage is characterised by two concepts:
• Server virtualisation
• Pan-array virtualisation
Even if customers do not see those as connected, they coincide & resonate and define how IT is deployed. And clearly builds on the most up to date Gen 1 arrays.
The ability to deploy many virtual machines each posing their own demands on storage resources is causing problems in the data centres. With workloads and virtual machines being portable across physical servers, the implications on storage become unpredictable. Add to this the desire to orchestrate DR provisioning for virtualised workloads and storage R&D designers are currently faced with new demands from customers.
It is also interesting to note that customers may feel tied into their server hypervisor vendor, but they have total freedom regarding their hardware provider. In contrast, on the storage side most customers are tied in to their hardware provider and the storage virtualisation schemes that come with the arrays.
Customers with capacity storage farms preside over numerous powerful array controllers. Increasingly they express that they want to be able to control all storage as a pool of tiered capacity irrespective of vendor make and models. We believe that ideally the authority to control storage lies in what we call a storage hypervisor. Here we examined the storage hypervisor philosophies. However some vendors have articulated alternatives which we think are less powerful because they are not heterogeneous. HP and EMC, are both promoting such an alternative called federated storage which is pegged at high-end enterprises. Nevertheless, even lower-end solutions will evolve such that arrays will share processing across arrays or be aware of other arrays in the same clusters.
The storage virtualisation outlook
Developing smarter array controller clustering is not trivial and the storage vendors arguably have an open field in terms of the architecture philosophies they could adopt. We think that eventually customers will be faced with a choice between homogeneous and heterogeneous propositions. We are also of the opinion that pursuing heterogeneity is the clever way forward both for customers and vendors. Even better will be array control which is tied into server hypervisors in order to provide proper workload orchestration.
We estimate that only 10%-20% of current high-end disk array customers have deployed storage hypervisor control of their arrays. But we believe that more customers would have exploited those capabilities if they were available to them. Over the coming years more vendors will add pan-array virtualisation leading to markedly higher adoption by customers. As is common practice in disk array development, we anticipate that features developed in high-end arrays will subsequently find their way into mid-range arrays. Eventually many businesses will exploit the ability to control their storage pool arrays across distances. This evolution will be accelerated by the ability to deploy capacity hosted by cloud based service providers.
There is plenty of scope for improving the storage array. And we can even imagine some of the ingredients that would constitute the third wave of storage virtualisation. Look forward to new start-ups, scope for market disruption as old and new storage system vendors battle it out.
Image credit: Navek