Why cloud storage needs to be tiered, flexible, and elastic

It's crunch time 

Data storage requirements of enterprises are rapidly changing as technologies like data analytics, artificial intelligence, machine learning, blockchain, robotics and the internet of things (IoT) are influencing usage.

Enterprises need more storage and more compute power to store and process more data, and faster and quicker networks to meet user expectations of performance. The storage spend is now driven by workloads related to the deployment of new technologies. Gartner predicts that by 2024, 50% of the global unstructured data storage capacity will be deployed as software-defined storage. The storage management, data protection and valuation of data continues to be an important aspect for businesses of all sizes.

With the transformation brought about by cloud, many of the underlaying aspects of data storage are increasingly becoming less important and the line between on-premise storage and cloud-based storage has started to blur because the selection of storage is increasingly being made based on business growth requirements and not on where and how the data is stored.

Why cloud storage is better make

As we all know, cloud data storage services are delivered on-demand and enterprises can pay per use.

This advantage not only eliminates procurement and management efforts involved in on-premise storage infrastructure, but also gives agility, global scale, and anytime-anywhere data access. Storage capacity, security and durability are managed by the cloud storage provider, which also ensures applications’ access to data. Applications access cloud storage through traditional storage protocols or directly through an API. Many providers offer additional services designed to collect, manage, secure and analyze data at a massive scale.

Key factors in cloud storage design

Before adopting a cloud storage solution, enterprises should undertake detailed due diligence to understand the existing storage landscape and application profiles in the context of future data growth and modernization plans.

A thorough assessment of storage tiers is necessary to understand the data life cycle - how the data is created and stored in the primary storage; where it can be accessed more frequently and at a very high speed; when it is moved to the next tier of primary storage where it is accessed less frequently; and finally when it is moved to the secondary storage tier for less frequent usage. A detailed assessment of the storage data life cycle is critical to design the target-state cloud storage and identify any hidden cost and associated complexity.

To ensure that critical data is safe, secure, and available when needed, there are several fundamental requirements to consider. It is important to ensure that natural disasters, human error, or mechanical faults do not result in any data loss. The data needs to be ‘redundantly’ stored across multiple availability zones and regions.

Data availability and security can be achieved by designing the right balance of access control, retrieval times and encryption of the data at rest as well as the data in transit.

AWS is one of the industry leading cloud-storage solution providers with a wide range of cloud-storage solution offerings. There are many flavors of AWS cloud storage – block storage (EBS), object storage (S3), archival storage (Glacier) and storage gateway service (AWS Storage Gateway), which provides storage for files, volumes, and virtual tapes using EBS, S3 and Glacier and enables applications to access the storage through standard storage-protocols such as NFS, SMB and iSCSI.

To design an optimal and elastic cloud storage solution, it is important to understand each of these storage service and the scenarios for which they would work best. Some of the design considerations are:

• EBS is a durable block-level storage for use with EC2 instances, and is well suited for use as a primary storage for data that changes relatively frequently, for example, databases, filesystems or applications that require block-level raw storage.

• S3 is the best-known and most-used storage of AWS. It is well-suited for static web content and for storing data for large-scale analytics or backup/DR purpose.

• Glacier is a low-cost cloud storage that provides highly secure, durable, and flexible storage. It is well suited for offsite data storage, and for backup and archival.

• Amazon Storage Gateway seamlessly provides secure integration between on-premise software and cloud-based storage. It is used for corporate file sharing, and also enables on-premise data backup applications on AWS S3.