Qlik Research head talks Associative Engine, NLP and Data Swarm

The research team at data visualization vendor Qlik is at work not only refining its Associative Engine, but also advancing technology projects such as conversational analytics and multi-attribute data analysis.

In this Q&A, Elif Tutuk, senior director of Qlik Research, details the new technology projects her team is incubating for future products. Tutuk's research group has been responsible for a number of successful Qlik technology features that are now integrated into products, including the Qlik Cognitive Engine -- an AI framework -- and Associative Big Data Indexing, which frees users from having to load data in memory and creates associative indexes at the data source.

Tutuk also explains how her team is further advancing the concept of running Qlik's AI Associative Engine on livewired data that defines connections between data points and is not based on developer-defined parameters.

She then breaks down the latest Qlik technology projects in development, such as a multi-attribute, visual analysis capability known as Data Swarm, and incorporating natural language processing and voice technology into Qlik's visual capability. Qlik hopes projects like these will help differentiate it from competitors like Tableau, Tibco and MicroStrategy.

Editor's note: This transcript has been edited for length and clarity.

How does Qlik's Associative Engine help data livewire itself?

Elif Tutuk: The unique thing about [Qlik's AI] engine is how it can automatically associate the data values itself. We run algorithms on the data that look at the data values to determine what possible connections can happen.

The first advantage of using an associative engine is that the data defines itself, compared to a relational approach where a human defines the connections between the data based on pre-canned questions. From that perspective, I use the terminology that the technology livewires itself, versus a relational approach, which is hard-wired and defined by developers. When users ask questions, they don't have to follow a predefined path to be able to drill down into the data.

With the Qlik Associative Engine, we analyze the full data set, which is different than when you use a relational query-based tool. When you ask a question as a user, you've already filtered down the data, so the AI is only looking at insights in that slice of data.

The first advantage of using an associative engine is that the data defines itself, compared to a relational approach where a human defines the connections between the data based on pre-canned questions.

At Qlik, we are using the question as a way for the machine to understand your interests and set a context, but then we are not slicing the data. You get those additional hidden insights that help you to think outside of your initial question, and it helps you start asking the next question that you didn't even think of.

What other Qlik technology projects is your team working on?

Tutuk: One of them is a visual experience that allows the users to do multi-attribute analysis on data. We have been calling this Data Swarm.

As a user, I can tell the system what I'm interested in, but with my human mind, I'm limited in how many attributes I can come up with to analyze the data set. By using algorithms, we can help the user analyze their data and not be limited to one dimension, one measure or a couple attributes. The algorithms visualize the data as points and determine where each data point goes, creating a picture for the user to actually see the patterns and outliers in the data.

Based on the patterns that you see, you can further drill down on select data and do further analysis with more traditional visualizations, like bar charts and scatter plots. [This is] not in the product at this time; it's an example of how we're exploring capabilities for analyzing large volumes of data with many attributes.

How does conversational analytics factor into your research?

Tutuk: We have recently acquired CrunchBot [AI Inc.] to bring natural language capabilities into the Qlik Cognitive Engine. At Qlik Research, we have been already working on this conversational analytics experience.

I'm a big believer that just having a search box is not going to help users. You use Google when you already know the question that you want to ask. Search engines are great for those cases because you get the answers very fast. That's why we are building a search engine with more natural language capabilities.

But the conversational analytics experience that we are incubating right now is combining the search and natural language capabilities with a visual capability. Imagine an experience where you ask a question, you get an answer and then when you ask the next question, we actually help you visually see the paths that you are going through. You can literally see your train of thought. It's a really new approach to thinking about search and BI.

With the amount of data that we have, you really need to create an experience where the user can see multiple visualizations at the same time and then interact with that. Then, as you interact with that, it gives the machine an understanding of your context and the things that you are interested in. That's on the roadmap and we already have a working prototype.

How will voice search play a role in future Qlik technology?

Tutuk: Voice has its place. If I were a salesperson driving to my next appointment, I just would want to ask my question with voice. That's one of the things that we would like to achieve in the Qlik Insight Bot, which is how we are going to rename CrunchBot. You will be able to have voice commands in addition to typing.

There are different users with different needs, but they all have the same expectation -- they want to use the technology easily and they want to be able to be augmented to get to actionable insights. That's what we are working on now at Qlik Research.