Sergey Nivens - Fotolia
What is 'ground truth' in AI and deep learning?
'Ground truth' is the bedrock of AI and deep learning. Carnegie Mellon professor Tom Mitchell explains the term and its significance with an example from healthcare.
"It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts." -- Sherlock Holmes, from Arthur Conan Doyle's "A Scandal in Bohemia"
Although Sherlock Holmes was no doubt talking about human intelligence's propensity to twist facts to fit theory, fast forward 127 years and the same can be said about AI and deep learning. The predictive algorithms that give machines the ability to analyze data are susceptible to similar twisting if they are not provided with high-quality data about the real world -- and lots of it. This "ground truth" upon which deep learning's convolutional neural networks depend usually requires humans to collect and provide the machines with highly reliable information. Creating such data is not trivial -- it not only requires abundant data for the classifiers in the computers, but also "test" ground truth data to make sure the artificial intelligence has correctly learned from the ground truth that was provided.
Tom Mitchell
We asked machine learning expert Tom Mitchell to give us his take on ground truth. Mitchell is the E. Fredkin University Professor at the Carnegie Mellon University. The former chair of CMU's Machine Learning Department, he is known for his seminal work in machine learning, artificial intelligence, and cognitive neuroscience and is the author of the textbook Machine Learning. He is a member of the United States National Academy of Engineering, a fellow of the American Association for the Advancement of Science and a fellow of the Association for the Advancement of Artificial Intelligence. For an interesting (data-based) take on the workforce jobs that machine learning systems, as currently configured, can and can't do, check out this article Mitchell co-authored with MIT professor Erik Brynjolfsson in the December issue of Science Magazine: "What can machine learning do? Workforce implications."
What is ground truth in AI and deep learning?
Read on for Mitchell's explanation of ground truth:
Ground truth is just another word for truth. For example, if we are interested in training a machine learning system to classify images of skin lesions as cancerous or not, we can think of two ways of collecting training data:
- Ask a good doctor to label each image according to whether they believe it is cancerous.
- Wait for the biopsy of the patient, which gives the true answer (also referred to as ground truth).
That's it. One reason this is important is that if we train on data of the second type (ground truth), then we have a possibility to outperform the doctors themselves. If we instead train on what doctors think, we can, at best, learn only to mimic the doctors.
