Conversational UX design: What it is and who's paving the way

To help advance the conversational abilities of AI systems, IBM isn't just relying on technologists. The inventor of Watson also turned to two experts with backgrounds in sociology and art.

Bob Moore has a Ph.D. in sociology and has worked in the research departments at Yahoo Labs and Xerox. Raphael Arar has a Master of Fine Arts and is an award-winning designer, artist and researcher. The two of them have spent the past few years at IBM applying their respective knowledge of conversation analysis and user experience design to help build more human-like conversational systems.

Together with their team, Moore and Arar are at the forefront of an emerging design field: conversational UX design. The field -- which bridges conversation analysis, UX design and AI -- aims to address a major deficit in many current conversational AI agents: the ability to string together bits of natural language into a conversation that approximates how humans talk to one another.

In part one of this three-part interview series, Moore and Arar explain what conversational UX design is and how their research -- and 50 years of sociology literature on conversation -- is contributing to the evolution of conversational systems. Their goal: To create a set of standards for designing conversational systems that others in the field can use.

Editor's note: This interview has been edited for clarity and length.

What is this emerging field of conversational UX design? What's the main objective?

Bob Moore

Bob Moore: Let me first make a distinction, because conversational systems have been around a while. There's a current technology that's popular, but then there's also a coming technology. The kind of system that we've been focusing on at the moment is what's been made popular with platforms like [Amazon] Alexa, Google Assistant, [Apple] Siri, [Microsoft] Cortana and some of the other startups. That's a version of conversational technology in which there's a design element. It's not fully machine learning based and it's not a full learning system.

I think we will probably get to a point where the technology shifts and most people are doing a learning-based system. In that scenario you wouldn't necessarily have a designer, as such, but more of a teacher or a trainer. I think that's the future, but right now, we're not quite there yet. Right now, that's still a hard problem. The platforms we have -- the ones I mentioned -- use machine learning to recognize what the user said, but then they require a designer to figure out what to say in response, in most cases. So, that's where a design role is needed -- [to figure out] how to respond to what the user said and how to classify what the user said so that [the AI agent] can respond in different ways. That's where we are right now.

These powerful new platforms were put in the hands of the masses and we have a lot of companies, large and small, experimenting with different kinds of systems and user experiences. So, I think there's a new discipline emerging here in which we're in an experimentation phase where there's not much consensus on how you design a good conversational user experience, and that's exactly the problem that we're trying to address.

Raphael Arar

Raphael Arar: To add on to that, we're really thinking about what's the process and methodology for how you approach and design [these conversations]. And so that's where our collaboration has really kicked off, because user experience design has, especially in recent years, instilled some solid processes for how to approach design problems.

One thing that [Moore] and I have been working on is trying to come up with a set of best practices and guidelines to teach others how to approach designing for these systems and how to do it in a methodical way. Right now, there's really a variety of techniques and resources -- or lack thereof -- so we're trying to bridge that gap.

How are you bridging that gap?

Moore: Part of what we're doing in [bridging that gap] is creating a library of UX patterns for conversational interfaces. The sociology part of this field of conversation analysis has a literature that's over 50 years old and the literature is filled with patterns of how people naturally talk to each other in a variety of different settings and languages. It's a treasure trove of patterns of conversation. What we've been doing is taking those patterns and using them to inform the design of UX patterns for conversational interfaces and then putting together a library of such patterns. If you're going to have a UX design discipline, you need to have pattern libraries and conventions, and so we call [our library and conventions at IBM] the Natural Conversation Framework. It's a set of UX patterns that are derived from this conversation analysis literature.

Part of our task here is to make the patterns in that literature accessible to UX designers. Because conversation analysts write for other social scientists, sociologists, linguists, etc.; they aren't concerned with building technology. So, the papers and materials haven't really been interpreted for [UX] design. We're trying to do that translation of taking the literature from that field and making it accessible and relevant to UX designers.

How close are we to having universal standards around conversational UX design?

Moore: I think we're a ways away.

Arar: That's actually a really good question. If you look at the discipline, you see a lot of common themes now, but there are different players with their own set standards or guidelines.

We know how to tailor our speech to one another in a way that these designed experiences try to do. So, at a broader level, we already have the guidelines [for conversational UX design], because we know how to hold conversations as humans. At a more granular level, it's going to take some time.

The nice thing about what's emerged is that there's now a consensus that human-centered design and designing with empathy for your end user in mind is a common theme.

Moore: I think what a lot of people are doing right now is they're using common sense [when designing for these AI systems]. The one advantage of these current platforms is most of them have an interface or an offering tool so that nonprogrammers like myself -- sociologists and anyone else -- can create a conversational user experience or a conversation space. That's great and that's why it's opened up [conversational experiences] to the masses.

But then what happens is we figure, 'Well, we all know how to talk and we all know how to have a conversation, so let's just build [a conversation space].' It's easy enough when you're identifying questions and writing answers, but … once you really get into it, you find that it actually gets pretty complicated. Even though we know how to do it, the average person isn't really good at articulating exactly how we hold a conversation or describing the machinery of human conversation. That's exactly what the field of conversation analysis is all about; taking those mundane, very familiar practices of conversation and formalizing them and specifying them in detail.

I think, given time, people can come up with this [on their own]. They know how to do it and they can observe it. But conversation analysts have been observing conversation for 50 years and have developed a conceptual framework and a set of findings around that. Why not start with that instead of reinventing the wheel? We're trying to jumpstart that with more formal knowledge of how conversation works.

Continue onto part two, "How to make AI agents better conversationalists: Context is key", which delves into Moore and Arar's work on training AI agents to understand context.