Recipe for lost customers: Flight cancellations and AI algorithms

Algorithms take the fall for customer service issues

Many airlines employ algorithms, or black boxes of rules to determine who's denied boarding in sticky situations. A different kind of algorithm, the training algorithm, serves as the backbone for setting the next-generation CRM artificial intelligence (AI) systems loose on enterprise data stores.

Both types of algorithms heavily influence -- or will soon -- the quality of customer experience.

Deep behind the systems administering CRM for airline service, retail businesses, B2B suppliers and even companies using it for human resources, there are data scientists tuning the algorithms and setting the rules they follow. Somebody had to write the rules for the black boxes.

While automation is attractive, algorithms -- whether they run rules engines or are training AI systems -- aren't people. The strength of the technology becomes a weakness when a whole group of customers inconvenienced by an algorithm walk away from your company.

The strength of [AI] becomes a weakness when a whole group of customers ... walk away from your company.

"We need to get back to real-world values again, and look at where software algorithms increase productivity without crossing the line of using math models and computer code strictly for profit," said Barbara Duck, who blogs about health data and worries about patients getting lost in the system or, worse yet, being unfairly tagged with medical problems and costs that insurance companies' predictive analytics systems uncover about patients, their customers.

Eight years ago, Duck half-jokingly suggested the U.S. government create a Department of Algorithms to protect patients and consumers.

"We've made that big mistake in healthcare with everyone taking in [flawed] prediction models versus working with cold, hard facts relating to the patient's health," she said.

Buyer beware: Discriminatory AI robots carry legal risk

In some scenarios, bad AI algorithms can bring legal risk, as well.

On the HR side, Christopher Fluckinger, who earned his Ph.D. in industrial and organizational psychology, and who happens to be my nephew, said algorithms may pose dangers to a company if a technology's data output is used too close to the decision-making process before it's vetted by humans.

Algorithms for screening job applicants absolutely cannot be one size fits all. They have to be customized in valid and nonsuperficial ways, said Fluckinger, an instructor at Bowling Green State University's Firelands campus in Huron, Ohio.

"A lot of assessment and candidate selection does involve a great deal of automation, aggregation, transformation and standardization of data," he said.

And Fluckinger ought to know, having concentrated his research on how to apply those technologies in the context of fairness and legality, as well as including human decision-making in the process.

In discussing these technologies with vendors at conferences he's attended, Fluckinger said salespeople selling HR systems rarely know much about tuning and validating algorithmic output to keep it useful, but nondiscriminatory. In his experience, salespeople often don't have the technical background to explain to customers this often expensive and time-consuming process, which leads to shaky or outright false claims about what these technologies can do.

Even worse than glossing over the difficult process of properly tuning algorithms, Fluckinger added, is when vendors cut corners in software development.

"The software side [of a technology vendor] will initially partner with test designers/validators, and then decide they can do the whole thing themselves," he said. "The end product looks basically the same and will make sales, but, of course, doesn't really work, and will be in shaky legal standing if challenged."

The ethics of AI algorithms

Some companies may believe that using algorithms for difficult decisions helps create fairness -- or the illusion thereof -- for consumers caught in situations such as overbooking or flight cancellations. That's wrong.

Alan Trefler, CEO of Pegasystems, held a question and answer session for media and analysts at the PegaWorld user conference in Las Vegas in June. He made the case that companies should consider monitoring the opacity of their AI systems; the more transparent the system, the more visibility companies have into how the machine trained itself to make decisions. The more opaque, the harder it is to figure out how the system arrived at a particular decision.

When AI does things like create Rembrandt-esque paintings, opacity is fine; no one really cares why the AI chose to make a particular brushstroke, Trefler said. But when it's supporting customer interactions in regulated industries -- for example, helping banks decide who gets a loan and who doesn't -- transparency becomes essential to make sure the AI cleaves to company values, as well as regulatory mandates. In fact, it should be a matter of the corporate record.

Anthropomorphizing algorithms as fair and benevolent decision-makers ... can lead to a bad customer experience.

Some customer engagements can be trusted to AI, Trefler continued, such as checking for markers of fraudulent activity. But the more sensitive the engagements are, the more they require human intervention.

"Having artificial intelligence that would sense when a particular decision was appropriate or not would require AI to have a sense of ethics, which I think is well beyond the level to which we should trust it today," Trefler said. "We need to, ourselves, govern it according to ethical standards."

Most companies probably won't get into such heady ethical discussions. They're more interested in bottom-line customer retention and finding CRM automation that can improve it. The one lesson from Trefler that companies should think about: Anthropomorphizing algorithms as fair and benevolent decision-makers, and letting front-line workers throw up their hands and blame the computer, can lead to a bad customer experience.

Certainly, for me, when it comes time to click OK on my reservations next time I'm headed to downtown Toronto for business or pleasure, I'll have to make a choice: Air Canada to Pearson or Porter Airlines to Billy Bishop.

Having gone through what I did, the company that earns my slim slice of the company travel budget spend will most likely be one whose AI algorithms haven't yet left me scrambling for an Uber after 11 p.m., standing alone with my luggage on the sidewalk outside Billy Bishop, employees streaming out for the night heading home. That's how I'll vote -- with my wallet.