In Kubernetes management, infrastructure work is tricky
While Kubernetes has a lot to offer when it comes to deployment and container management, developers still need tools that help with infrastructure management.
Container use is on the rise, and Kubernetes has emerged as the primary management approach. But IT teams face challenges when they try to incorporate container management functions into applications.
New technologies emerge in a manner similar to building a house. To get the structure in place, designers need blueprints and standard interfaces that ensure, for instance, that water runs when a person turns on a faucet. When they build containerized applications with Kubernetes, developers spend a lot of time on infrastructure grunt work, similar to putting in the plumbing. And there's growing consensus that there ought to be a better way to handle those Kubernetes management tasks.
"Kubernetes is a great system for deploying and managing applications at scale, but it doesn't really do very much to help you build the application in the first place," said Brendan Burns, an original Kubernetes developer and an engineer at Microsoft.
Digging into system infrastructure innards
To connect different systems, programmers work with obscure interfaces and have to learn the syntax for different development tools. As a result, they spend a lot of time on the configuration of system infrastructure rather than on fine-tuning the applications that run on top of it. Not only is the Kubernetes management work cumbersome, it is also repetitive.
"If you have an application for multiple customers, you repeat the infrastructure development work every time you deploy the application," said Joe Duffy, co-founder and CEO at Pulumi, a cloud development platform supplier.
At the Cloud Native Computing Foundation's KubeCon conference in 2017, Burns, who helped Google develop Kubernetes before turning it over to the open source community, introduced an experimental project called Metaparticle. Its intended mission is to simplify the grunt work.
While the Metaparticle initiative has been off to a slow start, it -- or something similar -- is eventually expected to address challenges IT professionals face to synchronize data updates across multiple systems.
"Metaparticle acts like building blocks on top of distributed computing systems managed with Kubernetes," explained Rhett Dillingham, senior cloud services analyst at Moor Insights and Strategy.
Rather than dabble with low levels, the Metaparticle project adds an abstraction layer, which acts like a virtualization layer in servers. The extra layer reduces the translations that developers must write to get their applications to run. Metaparticle would address the complexity that IT teams confront when they set out to package apps in software containers, replicate them, share them and synchronize their work.
As an open source project, Metaparticle aims to democratize the development of distributed container systems. The building blocks enable novice distributed system developers to incorporate proven development techniques into their applications. For seasoned developers, these libraries can be a tool to help them build applications more quickly and more reliably.
An idea with potential
Clearly, there's a problem here that needs to be solved.
"When I first heard about Metaparticle, I thought the idea was very cool," said Torsten Volk, managing research director at Enterprise Management Associates.
The development of such a framework has the potential to lower the bar for the creation of cloud container applications. This would be similar to open source projects that emerged in other development areas, such as web and mobile applications. Because a great deal of programming expertise is no longer required to build those, many non-technical business managers can now work on them.
If you have an application for multiple customers, you repeat the infrastructure development work every time you deploy the application.
Joe DuffyCEO at Pulumi
Indeed, a number of initiatives have begun to percolate in various open source communities as well as from within established and startup vendors.
Also coming along are new programming languages, such as Ballerina, which is intended to simplify the creation of microservices for API integration.
Chef and Puppet, meanwhile, can add Kubernetes automation features. The Docker Foundation integrates Kubernetes support into Docker Desktop. Google promotes App Engine. Startup container developer tool suppliers, such as Pulumi, are focused on the automation of cloud infrastructure development tasks.
But the road from cool idea to widely accepted standard often meanders; sometimes it leads to a dead end. Initially, Burns' star status in the developer community and Microsoft's leading industry position garnered the project a lot of attention.
But to be successful, such work needs a groundswell of support from developers. If that step takes place, the idea then needs to be picked up by a group that becomes responsible for its care in both the short and long term. For instance, the Kubernetes project eventually fell under the auspices of the Cloud Native Computing Foundation. The use of Kubernetes management has grown considerably.
While Burns' idea garnered attention, support has been minimal. And there is no assurance that the work itself will ever gain widespread acceptance.
"Most open source projects die on the vine," Moor Insights' Dillingham noted.
Timing, too, may be a problem for Metaparticle. "Sometimes, good ideas are introduced a bit ahead of the marketplace," said Enterprise Management Associates' Volk.
What next?
Burns understands as well as anyone that his project is in an embryonic stage. "I hope that Metaparticle is one statement that is influencing how we will build cloud applications in the future," he said. "But I think right now, it's really exploratory. I want to get people with similar ideas to come and work with me and explore together. ... Right now, it's a pretty small open source project, so I'm not too worried about standards. Perhaps we will influence future standards that someone may write, but that's a long way off."
It seems likely that Metaparticle, or something like it, will take hold to simplify things. Whatever form that something ultimately takes will be welcomed by IT professionals who deal with the complex issues related to the integration of Kubernetes management with new cloud-native applications.
