Deephaven extends real-time streaming data platform
Deephaven Data Labs has built a platform financial services organizations use to operationalize real-time data for queries and historical reporting of financial and client data.
Real-time data platforms are increasingly in demand as organizations aim to use event data for business operations and analytics.
Among the vendors in the industry is Deephaven Data Labs, which released an updated version of its data platform on Nov. 2 that provides users with new capabilities to publish Kafka data streams.
Deephaven, based in New York City, has both enterprise and community editions and provides a database platform for users to consume event streaming data. The Deephaven platform integrates a data query engine that enables users to analyze and use the data.
Among the primary applications for Deephaven is in financial services, which is where independent equities exchange MEMX uses the technology.
Craig Smith, head of development at MEMX, said Deephaven is an integral part of the organization's data environment. Data in the Deephaven data store is the source of information for MEMX and its clients' queries.
"Deephaven's ability to rapidly ingest data and make it available for processing and querying enables us to provide near real-time monitoring capabilities to our operations staff," Smith said.
Powering real-time event streaming and historical queries
While real-time data queries are a core part of MEMX operations, the exchange also needs to query historical data.
Smith said Deephaven provides the ability to merge intraday data into daily and historical data stores. He also that MEMX can query data in the Deephaven data store for reporting for internal applications, regulatory compliance and billing, and MEMX clients query the Deephaven data store for information related to their orders.
The new support for publishing Kafka streams interests Smith as well, he said.
Deephaven's real-time streaming data platform has a developer environment that enables users to build data queries.
"The ability to make that distilled data available via Kafka streams could further expand our ability to define and extend our existing data pipelines," Smith said.
Deephaven designed for real-time streaming data needs
Pete Goddard, CEO and founding partner at Deephaven, said the vendor has built a data engine to meet the high-performance needs of users in financial services and other industries.
Deephaven was originally built as an internal project at a Walleye Capital, which Goddard co-founded.
Deephaven's ability to rapidly ingest data and make it available for processing and querying enables us to provide near real-time monitoring capabilities to our operations staff.
Craig SmithHead of development, MEMX
Goddard explained that his team at Walleye needed a high-performance data platform and decided to build one on their own.
In 2016, Walleye spun out Deephaven as a separate business.
In May 2021, Deephaven began the public development of its Deephaven Core community release. With the community release, Deephaven is hoping to engage and grow a community of developers interested in contributing to and using the platform.
One of Deephaven's goals is to make it easy for developers to extend and integrate with the platform. The data platform is a Java application that Goddard said has tight integration with Python and C, the widely used programming languages for data engineers.
While support for publishing externally-facing Kafka streams is a new addition, Deephaven already supported integrating inbound data streams from Kafka.
Goddard noted that Kafka is widely used as what is known as a pub/sub messaging system, in which publishers publish data streams and users subscribe to them. What Deephaven is providing now is a way to do both publication and subscription as well a data engine for querying the event data.
Goddard said Deephaven is looking to build out more integration capabilities for the platform, including change data capture sources of data.
_mobile.jpg)
