Instaclustr to support Karapace open source schema registry
Instaclustr is joining rival Aiven in an effort to support an open source technology that brings REST proxy and schema technology to Kafka event streaming.
Database-as-a-service vendor Instaclustr said it is now supporting the open source Karapace schema registry technology, which was started by rival vendor Aiven.
Helsinki-based Aiven is also a database-as-a-service vendor and supports multiple database and data technologies. Among the services Aiven provides is managed Apache Kafka.
In 2020, Aiven launched the open source Karapace project, which provides a schema registry for Kafka event streaming data as well as a REST, or REpresentational State Transfer, interface.
Instaclustr, based in Redwood City, Calif., revealed on March 10 that it is now also backing the Karapace technology by supporting the open source project with code contributions. Instaclustr also plans to provide commercial support service for Karapace as part of its own managed Kafka offering.
Instaclustr said it plans to make that commercial support generally available by the end of this month.
Karapace has a growing number of users on the Aiven platform -- among them is online travel service Priceline. Anan Garg, director data engineer at Priceline, said that in his view, it will be helpful for both Aiven and Instaclustr to support Karapace.
Priceline currently is not using the REST proxy capabilities that Karapace provides, but plans to use them in the future to capture data directly from front-end web and mobile interfaces in real time, Garg noted. However, Priceline is using the schema registry capabilities on the Aiven platform.
[Karapace] schema registry is being used extensively in Priceline, by various applications, to validate schemas for the data being ingested to data lakes.
Anan GargDirector data engineer, Priceline
"[Karapace] schema registry is being used extensively in Priceline, by various applications, to validate schemas for the data being ingested to data lakes," Garg said.
Karapace brings open source schema registry to Kafka
The goal of the Karapace project is to provide an open source alternative to the Confluent Schema Registry. In 2018, Confluent changed the license for its Schema Registry to its own Confluent Community License, rather than the Apache open source license under which Kafka is available.
Aiven has aimed to keep the Karapace project compatible with the Confluent Schema Registry. With Instaclustr's help, Karapace is also now improving that compatibility with integrated support for Google's Protobuf, or protocol buffer, message format.
Instaclustr has been supporting Confluent's Schema Registry and REST Proxy as part of its managed Kafka service. But Instaclustr identified features that customers wanted to use that were not available under an open source license, said Paul Aubrey, vice president of product management at Instaclustr.
"By now collaborating with Aiven to add support for the Protobuf message format, we have been able to cement the Karapace project as a viable and fully open source alternative," Aubrey said.
Aiven and Instaclustr look to extend Karapace open source
Aubrey emphasized that Karapace has been designed as a drop-in replacement for schema registry, so the functionality is the same.
"By adding support for Protobuf, we are providing the open source community with extended flexibility," Aubrey said. "For developers and organizations using Kafka along with schema registry, this means no longer needing to make a tradeoff between functionality, licenses and being able to partner with a managed services provider like Instaclustr."
Soumya Bijjal, director of product management at Aiven, said Karapace has been popular with the vendor's customers. She noted that Aiven's plan is to continue to ensure compatibility with Confluent Schema Registry while also building new features in Karapace.
"All of us at Aiven are excited to see Karapace gaining traction in the open source community, especially Instaclustr's significant contribution to the code base with Protobuf support," Bijjal said. "Aiven is committed to helping this open source environment grow and evolve."
