Natural language processing drives conversational AI trends

Framing natural language processing

The term natural language refers to the manner in which humans communicate with each other. We use written and spoken forms of communication as well as gestures that have communication value. As such, we want machines to accomplish two goals: natural language understanding (NLU), which is machines understanding spoken, written and gestural communication, as well as natural language generation (NLG), which is giving machines the ability to communicate back to humans in their preferred language. Natural language processing (NLP) is the combination of these two aspects in systems that need to handle both directions of communication.

NLP is so core to the development of AI that it was one of the very sets of tasks that researchers attempted to tackle with intelligent systems, which is why conversational AI trends continue to be a hot of research and application development.

Chatbots and conversational agents

Chatbots and conversational agents were some of the AI applications to be developed -- MIT professor Joseph Weizenbaum created ELIZA in 1964 as a way to test the progression of realistic machine-to-human conversational interactions. Chatbots have evolved significantly from these early days but still are primarily text- or voice-based applications that respond back and forth to humans engaging in natural language dialogue.

Within the last decade, companies have looked to apply these chatbots and conversational interfaces to a wide range of applications: from customer support and assistant-enabled commerce to interfaces in automobiles and devices. Conversational interfaces reduce the complexity of the interface, provide constant service and support and feel more natural for human interaction. Simply put, people like to talk more than click or swipe.

Across a wide range of industries, companies and organizations have adopted chatbots and conversational interfaces. You can now find machine learning, AI-enabled conversational interfaces within hospitals as medical assistants, providing financial advice, providing educational support, interrupting your website experience to make sure that your needs are satisfied, and even providing automated conversations in game engines to make the environment seem more realistic.

Conversational interfaces are also finding their way into e-commerce and retail interactions. Rather than having to download a mobile app or scroll through pages of product options and offerings, conversational assistants give buyers a streamlined way of communicating their intent and finding the best products, services or configurations that meet their needs. In the future, we might find that we prefer conversational commerce over traditional methods that can lead to less-optimal purchases.

Voice assistant applications

Voice assistants are hardware devices that pair microphone inputs with audio output and AI to provide a range of intelligent capabilities. Voice assistant devices include Amazon's Alexa, Microsoft's Cortana, Apple's Siri and Home Pod, Google's Home and Assistant offerings, Samsung's Bixby, and additional offerings from companies like Baidu and others.

Voice assistants are tasked with a wide range of applications from simple music playing and home automation activities to much more complicated multistep conversations that involve keeping track of multiple parts of a dialogue. Enterprises and organizations of all types are looking to voice assistants to help with tasks ranging from customer support and guidance to augmenting human process activities. In the home environment, voice assistants are being used to help people with disabilities live independently. Paired with smart home appliances, these voice assistants can help with a variety of tasks such as turning on and off lights or adjusting thermostats.

Major brands are creating apps and skills on voice assistant platforms to interact and engage with users and customers. The Tide laundry detergent skill can help recommend how to care for hard-to-wash fabrics or how to use your washing machine. Domino's pizza skill allows users to navigate voice-only menus and place orders using the voice assistant. Virtual AI assistants can help gyms and fitness centers answer questions without involving the need for additional staff.

Hospitals are now experimenting with the use of voice assistants in patient care rooms to give their patients a better overall experience. Given that patients in hospitals may have limited mobility and be confined to bed, voice assistant devices help patients move their bed positioning, turn lights on and off, and call the nurse for additional assistance -- all with just their voice.

Analyzing sentiment and content

Sentiment analysis is the process of identifying and categorizing text in order to determine whether the person's attitude is positive, negative or neutral. While not usually thought of in the same context as natural language processing, sentiment, mood and intent analysis does form one part of the conversational and human interaction pattern. Sentiment analysis allows companies to analyze customer feedback to identify top complaints, track critical trends over time and gain a more complete picture of the voice of the customer. Sentiment is, in many ways, the emotional component of human conversation; sentiment only makes sense inside of human conversational or interpersonal interaction. Indeed, analyzing sentiment is important to understanding the intent of the person who is communicating.

Conversational systems are also using the power of natural language to extract key information from large documents. Instead of using NLP to simply provide conversational context and understanding, we can use NLP approaches to give machines a way to digest thousands of documents and summarize their main content components. AI systems are analyzing press releases, financial documents, business documents, email messages, voicemail, images, health records, contracts, mortgages, insurance policies, presentations and many other document types. The AI systems are finding detailed information in unstructured data and generating readable narrative from quantitative data. AI is also summarizing these large documents into shorter documents for use in other communication forms. Content summarization systems are even capable of generating "news stories" from social media and other data.

Content generation

Similar to content summarization, the conversational pattern also includes AI-enabled content generation, where machines create content in human language format either completely autonomously or from source material. Content generation can be done across a variety of forms including image, text, audio and video formats. AI systems are increasingly being used to generate breaking news content to bridge the gap until human reporters are able to get to the scene.

Reuters is using AI to scour Twitter feeds to find breaking news before it becomes headlines. The Washington Post Heliograf bot generated over 850 articles in 2017, covering rapidly changing news stories. AI systems are being used to generate sports content, especially for games reporters can't always be at such as all local and regional sports events.

AI is also being used to generate video and audio content. 21st Century Fox is using AI to generate movie trailers, highlight reels from sports games and other visual content. These systems can also assist with the of music soundtracks, background audio and even entire music albums. In 2018, Taryn Southern's album "I AM AI" was the to be completely produced and composed by AI systems.

Machine translation

The television and movie franchise Star Trek portrays the vision of the universal translator, where any language can be translated automatically to the native language of all parties. While we're still a ways away from this goal, AI is enabling machine translation. Machine translation combines aspects of NLU and content summarization with content generation to translate content between different languages. Machine translation falls in the conversational pattern, even though the end goal is to enable better human-to-human communication.

Machine translation provides access to information written or spoken in a language not native to the communicator, enables people to communicate with others who might not have native language proficiency, increases the reach of marketing and promotional content, improves the accessibility of support and help content, and provides content analysis of foreign language material.

One of the big challenges of machine translation is that language is culture and context specific, full of nuance and including slang, imprecisions and colloquialisms. This makes it difficult to faithfully translate the content and intent of something in one language to another. Facebook has been making significant waves here by utilizing a unique approach with unsupervised machine translation that can recognize the "shapes" of language contexts and the relationships between words to help make more faithful language translations.

Making AI-enabled conversations a reality

The conversational AI trends are just as foundational to AI projects as predictive analytics, pattern and anomaly recognition, autonomous systems, hyperpersonalization and goal-driven systems patterns. Like the other patterns, it continues to be a rich of research and product development. Soon, we'll be able to have the smooth conversations with machines that we see in science fiction movies, and with the rapid developments we're seeing, that day might not be too far away.