Amplitude adds A/B testing tool to analytics suite
Both the vendor's new feature as well as its recent acquisition of Command AI aim to make it easier for business users to work with its tools without requiring expert help.
Amplitude Inc. recently launched Web Experimentation, a feature that enables the digital analytics vendor's users to conduct self-service A/B testing without requiring data engineering expertise.
In addition, the company this week acquired Command AI, a startup whose tools provide AI assistance aimed at making it easier to use software platforms such as Amplitude's analytics suite. Financial terms of the acquisition were not disclosed.
A/B testing is important for data-driven decision-making, replacing intuition in product development rather than just the gut feel of developers and designers. It allows product managers to validate their ideas with empirical evidence.
Donald FarmerFounder and principal, TreeHive Strategy
A/B testing, a way to compare two or more versions of something to discover which performs best, is an important way for enterprises to determine strategy, according to Donald Farmer, founder and principal of TreeHive Strategy. In marketing, where Amplitude specializes, that might mean running tests on various campaigns or personalized recommendation algorithms to determine which is most effective.
"A/B testing is important for data-driven decision-making, replacing intuition in product development rather than just the gut feel of developers and designers," Farmer said. "It allows product managers to validate their ideas with empirical evidence. ... By testing [different] options, you can determine which works better for customers, ultimately leading to shipping the right product."
Based in San Francisco, Amplitude is an analytics vendor whose platform is geared toward digital marketing. Peers include vendors such as Mixpanel, Heap and Glassbox, as well as tech giants that provide marketing analytics tools, including Google and Adobe.
Recently, in September, Amplitude launched Amplitude Made Easy, a set of features aimed at helping new users get started with the vendor's tools.
New testing capabilities
Simulations such as scenario planning and A/B testing are valuable ways for organizations to determine the best courses of action and prepare for potential real-world situations.
Such testing, however, is complicated, generally requiring data engineers to develop pipelines and prepare data before tests can be run and decisions made based on their results. In addition, A/B testing can be a lengthy process due to all the work that needs to be done to ready data for running comparisons.
Amplitude, which was founded in 2014 and went public in September 2021, has provided A/B testing capabilities for three years. However, just as most A/B testing requires the involvement of experts, Amplitude's A/B testing capabilities didn't enable self-service analysis.
Web Experimentation, released Wednesday, aims to make A/B testing a self-service process.
With the feature's point-and-click interface that reduces the need to write code to execute tasks, self-service users can run experiments on their own or with minimal assistance from engineers. In addition, as its name suggests, Web Experimentation extends Amplitude's A/B testing capabilities to webpages and single-page applications so that users can understand which might be most effective.
As a result, Web Experimentation is a valuable new feature for users of Amplitude's analytics platform, according to Farmer.
"Web Experimentation is quite significant," he said. "The [point-and-click interface] will be good because it enables nontechnical teams to run experiments so that Amplitude can reach not just developers, but project managers and marketers as well. The self-service nature of this will no doubt be important. There are some advanced features in there too."
Beyond its interface, Web Experimentation includes the following:
A guided user experience designed to assist self-service users as they set up A/B tests.
A custom code editor that contains prebuilt blocks of JavaScript, CSS and HTML so that engineers don't have to write code for A/B tests from scratch.
Web Experimentation's basic capabilities are now generally available and can be accessed for free through Amplitude's Starter plan. A version that includes segmentation capabilities is available through the vendor's Plus plan, which starts at $49 per month. And a version with a code editing tool is available through Amplitude's Growth plan, the price of which is not publicized.
While Web Experimentation broadens Amplitude's A/B testing capabilities and aims to enable more employees within organizations to conduct A/B tests, the impetus for its development came largely from customer feedback, according to Jheel Jawharkar, principal product manager at Amplitude.
"Web Experimentation was our most requested experiment capability, so this is a natural, customer-led addition to Amplitude's analytics platform," she said.
In addition, it fits into Amplitude's belief that self-service analytics is an important means of making digital analytics a less complex and frustrating process for organizations, Jawharkar continued.
"Web teams have to be able to move fast, but experimentation is critical to them driving strong conversion and delivering personalized experiences," she said. "With Web Experimentation, they can leverage point-and-click interaction and visual editing so they can move fast without engineering support."
The acquisition
While Web Experimentation aims to simplify A/B testing so that self-service users can gain insights and streamline decision-making, Amplitude's acquisition of Command AI aims to make its overall platform easier to use.
Based in San Francisco, Command AI is a 2020 startup offering an AI-based assistance platform designed to aid users with AI-powered chats and automatically generated insights to help marketing and customer relations teams better engage with clients. Before its acquisition, Command AI raised $23.8 million in venture capital funding.
Once Command AI's capabilities are integrated into Amplitude's analytics platform, they will be used to provide personal assistance to users as they try to understand the behaviors of their own customers.
Given the potential benefits of AI chats and automatically generated insights, Amplitude's acquisition of Command AI is significant, according to Farmer.
In the grand scheme, it's just another established analytics or data management vendor acquiring AI capabilities, such as Snowflake buying Neeva or Qlik purchasing Kyndi, he noted. However, that doesn't make the acquisition of Command AI any less significant for Amplitude. And perhaps most important is the human capital Amplitude gains with Command AI co-founder and CEO James Evans, along with the rest of Command AI's employees joining Amplitude.
"The capabilities of Command AI are useful," Farmer said. "They extend the Amplitude platform and offer new features. However, the new development skills and a team with experience in AI are more important for Amplitude than any product features."
Amplitude expects to integrate Command AI's capabilities in early 2025, the vendor said. Features Amplitude expects to offer as part of its analytics platform using Command AI's capabilities include the following:
Guides and product tours that let users show off their own apps or websites in an interactive, digestible manner.
Smart triggers that enable Amplitude users to share product updates, event invites and special offers with customers.
Surveys so that Amplitude users can get feedback and generate insights from their own customers.
Checklists that enable users to get started with Amplitude by developing a personalized action plan.
Jawharkar noted that while Amplitude already provided the analytics tools customers need to gain insights, users requested more assistance as they navigate the platform and develop their own digital marketing tools with Amplitude. Acquiring Command AI aims to address their feedback.
"We heard that our customers wanted help encouraging the user behavior they wanted," Jawharkar said. "This is where Command AI comes in. And the best part is that everything Command AI does is stronger when combined with Amplitude."
Future plans
As Amplitude plots its roadmap for the next few months, the vendor's priority is blending Command AI's capabilities with Amplitude's existing analytics platform, according to Jawharkar.
In particular, there will be a focus on integrating Command AI's technology to enable more self-service use of Amplitude's tools.
"As we lean into the power of Command AI's offerings to broaden Amplitude's platform capabilities, we're excited to help companies improve their products and digital experiences in a way that acknowledges the uniqueness of every user," she said.
Farmer, meanwhile, said Amplitude is on the right track with its focus on adding more AI functionality and making its platform more accessible to nontechnical users.
He noted that Francois Ajenstat, who helped lead Tableau's product development for more than a decade and took over as Amplitude's chief product officer in November 2023, has a customer-centric focus. Seeing an effort to make the vendor's platform more customer-friendly, therefore, is not surprising.
Beyond integrating AI to better enable self-service use, Farmer suggested Amplitude could do more to help customers build predictive analytics tools.
"I expect to see more AI integration and user-friendly features like no-code or low-code development for digital product management," he said. "With Amplitude's vast data set and AI capabilities, they could also experiment with predictive product management by analyzing future user behavior and product needs."
Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.
