Andrea Danti - Fotolia
Hazelcast Jet 4.4 brings SQL to stream processing engine
Hazelcast Jet brings new SQL query capabilities to the stream processing platform that will enable developers to continuously query streaming data without interrupting data flow.
Streaming data has become an increasingly common and important part of modern data architectures. What hasn't always been easy, though, is the ability to query streaming data as it is being collected.
On Wednesday, Hazelcast announced the availability of its Jet 4.4 stream processing engine, which adds SQL capabilities to the technology. Hazelcast, based in San Mateo, Calif., has a number of products in its portfolio, including an in-memory data grid, as well as Hazelcast Jet. Alongside the 4.4 update, the company is positioning its combined platform as a data integration hub.
Mike Gualtieri, an analyst at Forrester Research, said the addition of SQL capabilities to Jet is a big addition. He noted that the new update will likely help to expand Hazelcast's position in the streaming data platform market.
"Jet SQL is a huge step for Hazelcast because it will greatly expand the scope of both use cases and users for Hazelcast," Gualtieri said. "Developers will be able to build real-time applications that query data in-motion."
How Hazelcast Jet 4.4 SQL enables streaming data queries
Scott McMahon, senior solutions architect at Hazelcast, explained that Jet ingests data that comes from different sources. The user then defines what processing action to do with that data with the Hazelcast pipeline API.
McMahon said that with the SQL capability, a developer can write a SQL query and submit that to run against any of the processed data event streams.
One use case for the SQL capability in Jet is for helping to enable fraud detection, according to McMahon. In that scenario, there is transaction card data flowing through the system as an event stream. An organization could choose to apply rules against the transaction data to provide a trust score for the transaction to determine if a given transaction is potentially fraudulent. He said that after the data has been processed, the typical next step is to send the transaction to an application for approval.
McMahon explained that with Jet 4.4, a developer can write a SQL query that could be attached to the data output stream. That query could be for specific attributes like location information, and the developers have the ability to start querying events right out of the data stream.
"With Jet SQL, it's basically a way for you to submit queries to existing streams running through the system, that you want to know certain things about," McMahon said. "But you're not interfering with the processing of those streams, and the query runs all the time."
Easier streaming data integration with Amazon Kinesis connector
As part of the Jet 4.4 update, Hazelcast is also releasing a new connector for Amazon Kinesis.
Kinesis is in many respects complementary to what Jet enables, according to McMahon: Kinesis is a messaging bus that enables streaming data delivery, he said, while Jet provides streaming data processing functionality. He added that Jet has had connectors for other data sources, including Apache Kafka, but did not previously have a native connector for Kinesis.
"So, now that's just another connector in our library, McMahon said. "If you want to connect to Amazon Kinesis, whether you're running in AWS or on premises or anywhere else, we can now just connect and treat that as a typical data source and just start processing, sending and receiving events in and out."
