BOSTON -- Cloud Foundry shops have reaped the flexibility benefits of cloud-native apps, but stateful applications and data services so far have been left behind.
Cloud-native apps, also sometimes known as 12-factor apps, are built around principles of a services-oriented, distributed approach to application design. The next frontier in cloud-native app development is how to incorporate data services and data management into existing cloud-native apps, and create a new breed of stateful cloud-native applications.
This endeavor will require improvements to application architectures and deployment processes, as well as updates to platforms such as Cloud Foundry PaaS, IT experts agreed during discussions at Cloud Foundry Summit here last week.
For example, Pivotal Cloud Foundry has already prompted new thinking around app deployment processes at CSAA Insurance Group in Phoenix, Ariz., but so far back-end databases and stateful applications have not been part of that revolution.
"Stateful applications and back-end databases are still painful to manage with our traditional IT ops stack," said Kyle Campos, DevOps transformation and cloud operations leader at CSAA. Campos said he'd like to bring the application automation and rapid development cycles that stateless apps get with Cloud Foundry to those stateful apps as well.
Cloud Foundry support for stateful applications varies by distribution. Pivotal Cloud Foundry supports tiles, which use Pivotal's Ops Manager and Cloud Foundry's BOSH to automate the management of stateful applications, such as MySQL, RabbitMQ, PostgreSQL and Redis. These packages only support certain configurations that correspond to the Cloud Foundry distribution's design. They also don't allow developers to manipulate and incorporate data services into custom apps through the Cloud Foundry Application Runtime.
Campos said he plans to evaluate tiles but isn't yet convinced that they're production-ready.
"Sometimes there's more smoke and mirrors [in tiles] than in the [Cloud Foundry] Application Runtime," Campos said. "You can only see if it'll work through experimentation."
For now, data services also are absent from the Cloud Foundry Application Runtime, which is focused on custom application development rather than back-end service modernization, said Chip Childers, Cloud Foundry Foundation CTO.
Container orchestration platform Kubernetes is maturing as a potential home for stateful services that offers more storage access flexibility than Cloud Foundry, but Cloud Foundry and Kubernetes integration is still in development.
(L-R) Mark Ardito of BCBS, Brian Dunlap of Southwest Airlines, Paul Puckett of Pivotal and Stephen O'Grady of RedMonk discuss cloud-native data services at Cloud Foundry Summit.
Stateful cloud-native apps require mindset shifts
In the long term, truly cloud-native applications that draw on persistent data services will require enterprises to refactor applications in ways that haven't yet been perfected, as well as transform their IT processes, according to a panel of IT industry experts.
"We have a data problem -- we've transformed the way we build apps, but data kind of got left behind," said Mark Ardito, divisional vice president of digital delivery at Blue Cross and Blue Shield of Illinois, Montana, New Mexico, Oklahoma & Texas (BCBS).
Modern data stores, such as Hadoop, and practices, such as streaming data through caching services to speed up performance, have helped, but organizational roadblocks persist, Ardito said. BCBS' database administrators (DBAs) have begun to put data repositories behind APIs to share data in a more service-oriented approach. But this also requires traditional DBAs to take on a reliability engineering role the same way IT ops pros have shifted toward site reliability engineering, and will take time.
The 12-factor app manifesto offers little guidance on how to redesign data services to be more nimble and shareable across sometimes ephemeral organizational and application boundaries. New approaches to data service architecture have emerged: caching, high-volume event logs to handle cross-boundary transactions, and metadata layers that front so-called 'dumb storage' systems. But none of these methods is a silver bullet, the panelists agreed.
I see people [in the industry] saying, here's the cloud, sign me up for [Amazon] DynamoDB, [Apache] Kafka and caching, and let's just have at it, but the real X factor is the conversation the organization has around that.
Brian Dunlapsolution architect, Southwest Airlines
"I see people [in the industry] saying, here's the cloud, sign me up for [Amazon] DynamoDB, [Apache] Kafka and caching, and let's just have at it," said Brian Dunlap, solution architect at Southwest Airlines, in the panel discussion. "But the real X factor is the conversation the organization has around that."
Not every so-called legacy idea should be thrown out when IT pros design stateful cloud-native apps, either, Dunlap said. For example, data schemas, which were deemed passé by some NoSQL database purveyors, shouldn't necessarily go out of style, even if they're as simple as agreed-upon date and time formats.
As enterprises incorporate data services into cloud-native apps, the conversation about 'shifting security left' will also apply where developers manage test data with potentially sensitive information, said panelist Stephen O'Grady, analyst at RedMonk.
"The way we manage data within organizations needs to change -- from a test perspective, there's a whole range of things we need to do to ensure, not only data integrity, but that it's being handled properly as well," O'Grady said. "That's one of the areas where we really need to see a lot of evolution."
