Comcast's monolith to microservices shift spotlights pains, gains
The shift to microservices is not an easy task. A senior software professional at Comcast explains the important lessons learned as his team moved Xfinity Home off of the monolith.
Organizations that want to move from a monolith to microservices architecture must prepare for the associated challenges and create the right foundation for this new architecture. Comcast software professionals learned this lesson firsthand.
Xfinity Home is a Comcast home security service that provides features such as 24/7 video monitoring and push notification alerts to mobile devices and has grown to incorporate home automation and connected services, such as remote climate control and door lock management.
"It became clear to us that our monolithic legacy architecture was not going to provide us the flexibility and agility we needed to serve our customers," said Tom Hughes, senior director of software and IoT platform engineering at Comcast. "Building a microservices platform was the obvious choice."
Lessons learned
Comcast is not new to microservices, as its X1 video platform was built on microservices from the ground up, Hughes said. However, this is the company's first major shift from a monolith to microservices, which required the software team to shift their perspective and processes. For starters, Hughes said, the team revisited a decision to build a cloud-agnostic architecture to quickly shift between various public and private cloud environments.
"We underestimated the complexity involved with implementing this decision and the impact that it would have on our velocity," Hughes said. "Once we pivoted to an approach focused on a single cloud environment [AWS] for this specific project, we realized the rate of development and advantages to leveraging components native to that environment."
The team also had to overcome "personal bias," Hughes said. They previously worked with monolithic applications that contained all required business logic, but they had to adjust their mentality for a microservices architecture and not overload services with excessive logic.
"It's sort of like retraining yourself to limit what you're trying to accomplish and what the purpose of a given service is and not making it more complex than it needs to be," Hughes explained.
I would still say that microservices is a double black diamond [ski] trail. You need to be reasonably proficient at architecture to even attempt to try it.
Anne ThomasVP and distinguished analyst, Gartner
Comcast's experience, while successful, demonstrates that companies should approach the transition from monolith to microservices with eyes wide open.
"I would still say that microservices is a double black diamond [ski] trail," said Anne Thomas, VP and distinguished anaylst at Gartner. "You need to be reasonably proficient at architecture to even attempt to try it."
Some of these proficiencies, Thomas explained, include the establishment of Agile development processes and DevOps practices. Hughes echoed this idea, saying it was crucial for his team to implement effective DevOps team structures and methodologies.
"It may be possible to build and operate a successful microservices architecture using a more traditional Waterfall development approach … but as I think about our own experiences, it would be really difficult to do so, and the finished product would suffer," Hughes said. "Knowing what DevOps team built a service or component, and is still on the hook if there's an issue, is extremely valuable in ensuring accountability and trust throughout a distributed environment."
One of the major requirements for microservices is that the platform provides basic microservices management capabilities, such as discovery and routing, as well as operational capabilities, such as automation and monitoring, said Gary Olliffe, research director at Gartner. Hughes agreed and added that the combination of automation and testing played a crucial role in the transition to microservices.
"We encourage all of our teams to test as they go, which means they find issues early and address them at the service level before those issues get folded into the broader architecture," he said. "Leaning on effective automation is a great way to build reliability and resiliency."
Prepare for complexity
While the new architecture has improved deployment times and updates to Xfinity Home services, it added a layer of complexity that organizations must prepare to manage, Hughes said.
"I think that some of the efficiency that you gain -- part of the trade-off -- is operational complexity, because there is that much more that you have to manage," he said. "It's great that services can be singularly focused … but when they are that way, there's logic that has to be implemented and spread across a larger array of services that have to be supported."
One of the biggest takeaways from the Comcast project was how important it is to create consistent tools and development environments for autonomous development teams, Hughes said.
"Standardize your core tools, SLAs [service-level agreements] and API contracts without focusing on languages or specific frameworks," Hughes said. "Ensure your entire team is starting from a common foundation with access to all of the tools and services needed to enable the autonomy of each of the development teams or squads, while still maintaining consistency."
Additionally, architects must adhere to the business goals that a microservices architecture is intended to serve, Olliffe said.
"The key is that architects must be clear on the business outcomes they must deliver and how they will measure them," Olliffe said. "Microservices architecture is not a goal; it is just a means to an end."
