Data ethics issues create minefields for analytics teams
Organizations have to map out ethical boundaries for data mining and analytics applications, according to participants in a roundtable discussion on data and ethics.
GRANTS PASS, Ore. -- AI technologies and other advanced analytics tools make it easier for data analysts to uncover potentially valuable information on customers, patients and other people. But, too often, consultant Donald Farmer said, organizations don't ask themselves a basic ethical question before launching an analytics project: Should we?
In the age of GDPR and like-minded privacy laws, though, ignoring data ethics isn't a good business practice for companies, Farmer warned in a roundtable discussion he led at the 2019 Pacific Northwest BI & Analytics Summit. IT and analytics teams need to be guided by a framework of ethics rules and motivated by management to put those rules into practice, he said.
Otherwise, a company runs the risk of crossing the line in mining and using personal data -- and, typically, not as the result of a nefarious plan to do so, according to Farmer, principal of analytics consultancy TreeHive Strategy in Woodinville, Wash. "It's not that most people are devious -- they're just led blindly into things," he said, adding that analytics applications often have "unforeseen consequences."
For example, he noted that smart TVs connected to home networks can monitor whether people watch the ads in shows they've recorded and then go to an advertiser's website. But acting on that information for marketing purposes might strike some prospective customers as creepy, he said.
Shawn Rogers, senior director of analytic strategy and communications-related functions at vendor Tibco Software Inc., pointed to a trial program that retailer Nordstrom launched in 2012 to track the movements of shoppers in its stores via the Wi-Fi signals from their cell phones. Customers complained about the practice after Nordstrom disclosed what it was doing, and the company stopped the tracking in 2013.
"I think transparency, permission and context are important in this area," Rogers said during the session on data ethics at the summit, an annual event that brings together a small group of consultants and vendor executives to discuss BI, analytics and data management trends.
AI algorithms add new ethical questions
Being transparent about the use of analytics data is further complicated now by the growing adoption of AI tools and machine learning algorithms, Farmer and other participants said. Increasingly, companies are augmenting -- or replacing -- human involvement in the analytics process with "algorithmic engagement," as Farmer put it. But automated algorithms are often a black box to users.
Mike Ferguson, managing director of U.K.-based consulting firm Intelligent Business Strategies Ltd., said the legal department at a financial services company he works with killed a project aimed at automating the loan approval process because the data scientists who developed the deep learning models to do the analytics couldn't fully explain how the models worked.
We've gone from a bottom-up approach of everybody grabbing data and doing something with it to more of a top-down approach.
Mike FergusonManaging director, Intelligent Business Strategies Ltd.
And that isn't an isolated incident in Ferguson's experience. "There's a loggerheads battle going on now in organizations between the legal and data science teams," he said, adding that the specter of hefty fines for GDPR violations is spurring corporate lawyers to vet analytics applications more closely. As a result, data scientists are focusing more on explainable AI to try to justify the use of algorithms, he said.
The increased vetting is driven more by legal concerns than data ethics issues per se, Ferguson said in an interview after the session. But he thinks that the two are intertwined and that the ability of analytics teams to get unfettered access to data sets is increasingly in question for both legal and ethical reasons.
"It's pretty clear that legal is throwing their weight around on data governance," he said. "We've gone from a bottom-up approach of everybody grabbing data and doing something with it to more of a top-down approach."
Jill Dyché, an independent consultant who's based in Los Angeles, said she expects explainable AI to become "less of an option and more of a mandate" in organizations over the next 12 months.
Code of ethics not enough on data analytics
Staying on the right side of the data ethics line takes more than publishing a corporate code of ethics for employees to follow, Farmer said. He cited Enron's 64-page ethics code, which didn't stop the energy company from engaging in the infamous accounting fraud scheme that led to bankruptcy and the sale of its assets. Similarly, he sees such codes having little effect in preventing ethical missteps on analytics.
"Just having a code of ethics does absolutely nothing," Farmer said. "It might even get in the way of good ethical practices, because people just point to it [and say], 'We've got that covered.'"
Instead, he recommended that IT and analytics managers take a rules-based approach to data ethics that can be applied to all three phases of analytics projects: the upfront research process, design and development of analytics applications, and deployment and use of the applications.
