Rockset brings Query Lambdas to real-time database
Real-time cloud database looks to help better enable developers to build applications with Query Lambdas capabilities that execute queries and lookups from REST endpoint triggers.
The concept of a SQL database trigger is one that has long been implemented in databases, enabling a specified event to launch or "trigger" a database action. Real-time cloud database startup Rockset is now taking that concept a step further with a feature it calls Query Lambdas.
Rockset in San Mateo, Calif., has its roots in the open source RocksDB key value store originally created at Facebook. Rockset provides a real-time database service that was updated on March 12 with the Query Lambdas feature, which can be triggered by events on REST endpoints to run developer queries. The Query Lambdas approach also provides a versioning system so every event is captured and integrates a secure transport mechanism to help mitigate risk from a potential SQL injection attack.
Among the early users of the Query Lambdas capability is Sequoia Capital, based in Menlo Park, Calif. Sequoia has also become an investor in Rockset, and uses the database to power many of the venture capital firm's internal-facing applications to help evaluate potential deals.
Hem Wadhar, a data scientist at Sequoia, said his firm uses Apache Spark to process its daily batch data pipeline, outputting the results to Amazon S3, where Rockset indexes and serves results to various internal tools. Wadhar said the Query Lambdas capability will enable Sequoia to write more front ends that can expressively query the company's data.
"Because the queries are versioned and can be exposed as API endpoints, we will both reduce our development iteration time and also reduce the round trip between an end-user request and data delivery," Wadhar said of the benefits that Query Lambdas will provide.
With Query Lambdas, developers can configure a REST endpoint to trigger a stored SQL query.
How Query Lambdas works with the real-time database
Because the queries are versioned and can be exposed as API endpoints, we will ... reduce our development iteration time.
Hem WadharData scientist, Sequoia
Databases are often at the foundation of modern applications, providing data that can be queried and delivered to users. According to Venkat Venkataramani, CEO and co-founder of Rockset, if you think about how people build applications on top of a database, it involves a lot of SQL queries and project lifecycle management. He added that more often than not, what developers are trying to build is an API or a service on top of data that is coming in real time.
"Query Lambdas are named SQL queries that are stored in Rockset that can be executed through a dedicated REST endpoint," Venkataramani said.
He added that Rockset users connect to data sources, which are automatically turned into SQL tables. With Query Lambdas, database administrators can now also store SQL commands that will automatically get triggered by events at a REST endpoint.
Looking beyond SQL database triggers
Venkataramani emphasized that Query Lambdas is more than just a simple database trigger in a number of ways.
"So, you can't really use a regular database trigger to serve data to your customers," he said. "Triggers are used to automate actions when some data you have comes into a table."
To illustrate how Query Lambdas provides more capabilities than a basic trigger, Venkataramani offered an example use case involving Twitter and a real-time JSON data stream that includes Twitter messages. In the example, a developer wants to build an API that returns the most tweeted-about stock ticker symbols in the Twitter data stream for a given period of time.
"Now you can basically build that application using a complex SQL query, you come into Rockset and save the query as a Query Lambdas," he said.
The developer then just needs to configure a REST endpoint that can be hit from any application, and will execute the query and deliver results back to the developer's application.
Query Lambdas provides versioning capabilities
Another key capability that Query Lambdas enables that isn't easily possible with a regular database trigger is versioning. Venkataramani explained that with Query Lambdas, for every modification done to SQL, the system will automatically generate a new version.
In light of that, an organization could run one version of a query in production on a real-time database in the cloud while testing other versions, with the ability to change versions as needed to optimize workflow. Versioning is also helpful for collaboration, as different teams within an organization can work together to figure out the most efficient queries without breaking a production workload.
"At the end of the day, we're all about developer velocity," Venkataramani said. "It's about how quickly you can go from a real-time stream, to building real-time applications and APIs on top of that."
