What chatbot evaluation metrics do you use to measure performance?

Chatbots are an integral part of a contact center's strategic customer care strategy, and any strategy requires measurement of success via data, analytics and metrics.

Many contact centers struggle with what chatbot evaluation metrics are most vital to measure and the importance of them, but the key is to break them down into a few categories and home in on what metrics you can use and what they say you about your service, business and customers.

It's important to understand your key business goals and objectives when establishing chatbot evaluation metrics, including overall goals and the performance improvements you are trying to make.

Performance metrics. This category focuses on the key metrics related to usage and interactions, such as:

• number of total active users;
• number of new users using bots;
• number of total interactions;
• direct or organic users;
• average daily number of chats handled by bot;
• average session or average handle time; and
• number of topics.

Customer experience metrics. This category focuses on the overall experience of the customer, including self-service, customer satisfaction and retention rates. Some of these metrics include:

• customer experience with the bot -- businesses can measure this by net promoter score, customer satisfaction or customer effort score;
• first-time users and retention rates;
• self-service resolution rate or goal completion rate;
• chatbot-to-human escalation -- these are considered unproductive interactions that need to be tracked for continuous improvement; and
• sentiment analysis score.

Financial/sales/growth performance. This category is focused on sales and cost, including:

• conversion rates -- chatbots converting a chat in to a sale;
• total direct chatbot sales -- customers using a chatbot to buy;
• total cost of chatbot ownership -- cost of operation and maintenance; and
• chatbot cost savings -- businesses can measure this by the percent increase of conversions, reduction in escalations.

These chatbot evaluation metrics can help contact centers measure overall chatbot performance in key areas to assess, evaluate and improve business outcomes.