At Logi Analytics and Tableau, beta feedback drives creation

Beta stage: The bread and beta-r of software development

As for Seattle-based Tableau Software Inc., when developing new products, "we think it's absolutely vital to have customers involved in every step of the process," said Adam Selipsky, Tableau CEO, in an interview during Tableau Conference 2018 in New Orleans.

It's a sentiment that, if anything, often manages to come out during the BI vendor's conferences, which see thousands of Tableau users and partners attend each year.

During the New Orleans event in October 2018, the vendor unveiled the public beta release of Tableau 2019, which added a number of AI-powered abilities and a new natural language processing component to the company's analytics and visualization tool. Users eagerly downloaded the beta, which is not set for a full release until early 2019, according to Tableau CTO Andrew Beers.

The public beta "goes out to anyone who wants to be in our beta program, which is all of the customers, right?" he said.

In the software world, the beta test stage falls toward the end of the software development cycle. It represents a working and fairly stable application or update, one that will work more or less as designed for many of the users who test it.

A lot of what we're looking for [during beta testing] is 'does this work in your environment, does this work with your data?'

For Tableau, beta feedback from its many beta users, those both small and more extensive, helps pinpoint bugs and solidifies the details of a new feature.

"A lot of what we're looking for [during beta testing] is 'does this work in your environment, does this work with your data?'" Beers explained.

"We try it on all kinds of different data sets inside, all different kinds of environments inside," he said, "but the variety of environments and data sets is basically infinite."

If the beta stage can be thought of as a goal, what's referred to as pre-alpha might be seen as a person's first few hesitant steps on that path.

In the beginning

Pre-alpha is generally thought of as anything that happens before the formal testing of a product. That could encompass things like concept development, initial requirements testing and software design. Most of what happens during pre-alpha is internal.

Things tend to open up during the middle phase of the cycle, the alpha stage. The core features of a product have likely been developed at this stage, but probably not much else. A product in alpha stage is liable to be unstable and fairly incomplete and, at least in Tableau's case, it can be highly malleable by the users who test it.

Alpha testing at Tableau is usually fairly narrow, Beers said.

"When we do an alpha, it's usually one particular team reaching out to a set of customers that they know."

In alpha testing, customers are able to try out new platform features many people don't even know exist and, in return, users provide information about where they run into errors, as well as ideas for new or improved features.

According to Beers, these early alpha users can have a lot of impact on development, as it's still "early enough that we haven't gone through any one-way doors and we haven't built enough of it that we feel bad going in another direction."

Quite often, a user's alpha feedback "does change the details of the direction" of a new feature, Beers said. Occasionally, while it's not nearly as frequent, feedback will change the course of the entire product.

While Tableau has multiple development teams and innovation teams, it can be difficult to correctly predict what new thing users need or want, Beers explained. Ultimately, it can come down to a trial-and-error process with planned or even partly finished products occasionally getting scrapped -- and their different pieces pulled apart and added to various other new products.

The development cycle, with its multiple tiers of user feedback, isn't only important and beneficial to Tableau, but it is also helpful for other BI vendors, including Logi Analytics.

Logi-cal beta testing

Privately owned and based in Virginia, Logi Analytics, formerly LogiXML, has a long history of providing BI tools since its launch in 2000. Focused on providing embedded, self-service analytics tools, the BI vendor entered the augmented analytics arena and introduced a new product, Logi Predict, which aims to offer users predictive analytics capabilities that are comparatively easy to use.

Apparently, the product concept intrigued users, as some 40 customers signed on as beta testers. According to Sriram Parthasarathy, senior director of predictive analytics at Logi Analytics, the vendor was originally hoping to get four or five testers.

Like the Tableau beta feedback, the Logi Predict feedback fueled ideas and innovation.

"It becomes a good way to not only get generic feedback about the predictive features," but also yielded feedback on specific issues and gave Logi Analytics some ideas as to how customers wanted to use the product, Parthasarathy said.

For example, the Logi team found that customers tend to say their data is extremely clean, "but when we actually played with the data, we realized the data is quite dirty and we had to clean it up," Parthasarathy said.

In doing so, the team discovered certain common dirty data problems across a particular industry, and so they optimized some of the automatic cleaning capabilities, Parthasarathy said.

However, Logi Analytics' attention to customer feedback doesn't end at the beta level.

To beta and beyond

Logi has fairly active user and developer communities with people frequently asking questions, suggesting ideas for new features, and giving each other tips and pointers on the vendor's forums.

"We constantly are creating new tools and updating the product based on customer feedback," Parthasarathy said.

Tableau also has an active user community.

"Most companies, you think it would be axiomatic," Selipsky said, referring to the practice of soliciting and listening to customer feedback.

"Actually, it turns out to be hard for most companies to listen intently, to listen at multiple listening posts, and then to take that feedback and ingest it and know how to do something with that," he said.