The challenges of cloud data management

Growing cloud management challenges

IT departments are facing a growing challenge to stay abreast of advancements in cloud technologies, provide day-to-day support for increasingly complex systems, and adhere to ever-changing regulatory requirements. In addition, they must ensure the systems they support are able to scale to meet performance objectives and are secured against unauthorized access.

Cloud data governance

A common goal for all organizations is to quickly identify and leverage data that delivers real business value. But statistics show that instead of focusing on strategic business objectives, many IT shops are now drowning in their own data.

Flexera is an industry-leading cloud management company that is known for its yearly "State of the Cloud Report." In its 10th year of publication, the 2021 report provides the survey results from 750 cloud decision-makers and users.

The 2021 report shows that, regardless of the organization's level of experience, data governance is one of the top four cloud challenges, with compliance being one of the most pressing issues for some shops.

Cloud data regulatory compliance

Of the Flexera survey respondents that described their organizations' cloud skill levels as advanced, 75% reported regulatory compliance was one of the most challenging aspects of cloud platforms.

Much like data security, adhering to regulatory compliance frameworks is a shared responsibility between the customer and cloud provider. Larger cloud vendors will provide third-party auditor compliance reports and attestations for the regulatory frameworks they support. It will be up to each organization to read the documentation and ensure the contents meet specific compliance needs.

Most leading platforms will also provide tools to help clients configure identity and access management, secure and monitor their data, and implement audit trails. But the responsibility for ensuring the tools' configuration and usage meet the framework's control objectives relies solely with the customer.

A couple of recommendations that will help include the following:

• Shared responsibility will vary by vendor, type of cloud service (IaaS, PaaS, SaaS) and implementation. The organization is ultimately responsible for meeting all of the compliance frameworks' control objectives -- not the cloud provider. A few examples of cloud vendor shared responsibility guidelines include the following:
  • Amazon Shared Responsibility Model;
  • Microsoft Shared Responsibility in the Cloud; and
  • Oracle Autonomous Database Responsibility Model.
• Although the vendors' documentation will provide a starting point, it does not provide a control-by-control responsibility listing. One best practice for cloud database management systems is to create a spreadsheet that lists each control objective, the entity responsible for compliance, the evidence needed and its location.
• Although a cloud vendor may provide compliance documentation for a given framework, that doesn't mean every product or service it offers is compliant. Most compliance reports will begin with a listing of the environments, applications and tools that are in scope.

Impact of cloud data growth on app performance

We know one of IT's core responsibilities is to transform raw data into actionable insights. That information must be presented to users in a timely manner. Rapid data growth that affects performance and the user's ability to tap into business intelligence in real time is becoming an ever-increasing challenge for cloud platforms.

Splitting data access between cloud and on-premises platforms can negatively impact performance due to data access lag. Increasing data volumes will continue to plague administrators tasked with supporting high-performance cloud applications. One of the primary drivers behind edge and fog computing is to move some of the compute and storage components closer to the devices and data sources the system interacts with to improve performance.

Many administrators currently rely on horizontal and vertical scaling to ensure good performance.

• Vertical scaling involves adding additional compute or storage resources to the server or replacing it with a system that has more power. You vertically scale a system in the cloud by upgrading to a larger instance size.
• Horizontal scaling means adding additional servers to accommodate workload increases. You horizontally scale a system in the cloud by adding additional instances.

Here are a few best practices to help you achieve better cloud performance:

• Plan -- a lot -- before deployment. Will your new application's workload scale predictably or do you need to accommodate processing spikes? If it spikes, you may want to evaluate your vendor's auto-scaling features. Vertical scaling won't scale as high as its horizontal counterpart. But you will need to architect your application in a way that fully leverages the benefits of horizontal scaling.
• Choose your vendor wisely. One of the benefits of a hyper-competitive cloud market arena is that it leads to a wide range of offerings that can meet virtually every processing scenario. Looking for a horizontally scalable, globally consistent, relational database service? Check out Google's Cloud Spanner. Do you need a world-wide platform that supports multiple NoSQL models? Microsoft Cosmos DB may fit your needs. Define your application's processing requirements and choose the vendor that best meets your needs, versus force-fitting it into one of your usual
• Conduct cost reviews during cloud provider selection and on a regular basis as the application matures.