Computer vision tools reach into test, healthcare, security

Eyeing commodity status

Beyond the current applications, computer vision tools are rapidly becoming commodified for lower-profile uses.

Totvs Labs, a software vendor based in Mountain View, Calif., has started using computer vision recognition in its product testing, especially its cloud-based software, which needs to be frequently updated and therefore continuously tested. Typically, a senior quality assurance engineer would run a test by writing a script that puts the application through its paces, and each test case would take hours to create, said Vicente Goetten, the lab's executive director.

Christopher Jones

If an application changed significantly because the framework and its underlying code needed updating, for example, thousands and thousands of test cases might have to be written, he explained. With the help of computer vision recognition, engineers can look at a website, desktop application or mobile app, rather than the underlying code, and see where they need to enter data or press buttons.

Totvs Labs didn't create this technology but instead purchased off-the-shelf tools that were relatively easy to deploy from machine learning testing vendor Functionize. Combined with other tools in the platform, Goetten reported a sixfold improvement in testing efficiency, and "we didn't have to increase our QA team."

[U]sing a neural network, you can actually learn from a smaller number of images.

Behind the scenes, AI-powered computer vision tools are playing a role in helping to spot phishing attacks. Email security software vendor GreatHorn Inc. wanted to add this functionality to its namesake product to help improve the accuracy of phishing detections and turned to open source tools.

"One of the systems we use is TensorFlow," said Christopher Jones, senior data scientist at GreatHorn. "We use convolutional neural networks, which are in the purview of deep learning."

The training data set includes login pages that Jones collected as well as several thousand screenshots the company had already collected. Even though that's a smaller data set than AI platforms typically need, by "using a neural network, you can actually learn from a smaller number of images," he said.

Healthy outlook for vision in healthcare

Mark Michalski

Radiology was one of the first areas in healthcare to take advantage of artificial intelligence, said Mark Michalski, executive director of the Massachusetts General Hospital & Brigham and Women's Hospital Center for Clinical Data Science in Boston.

"Back in 2016," he recalled, "the department of radiology at Mass. General kicked off this center and really focused on the task of, how do you use machine learning to enable diagnostic radiologies to interpret images more quickly, more accurately, more quantitatively? And since that time, the capability of machine learning has expanded considerably."

We're coming to a world where ... we have to think very deeply about how to protect the data.

Using AI in image recognition helps overcome the classic "black box" problem associated with other AI applications, according to Michalski. "With images," he noted, "'explainability' is a little easier, because you can provide heat maps over the tops of the images to explain why the machine is looking where it is."

Imaging technology has also matured to the point where radiology scans can be processed on systems that are within the hospital's control. Patient data doesn't have to be sent to a third party for processing. "Most of what we do is developed within our own firewall," Michalski said.

That's not the case for a lot of the other AI-powered medical tools. "We're coming to a world where it's very hard to make sure patients aren't identified based on data leaving the enterprise," he added. "So we have to think very deeply about how to protect the data."

Lauren Neal

Computer vision recognition is playing a role in a collaborative effort between a national medical research agency and Booz Allen Hamilton Inc. on the use of video and audio to evaluate discomfort levels of patients.

"You can tell by their word choice, or the expressions on their face, what level of pain they're in," said Lauren Neal, a principal at the IT consultancy.

And more applications are on the horizon. "There are a number of machine learning solutions that have been approved recently by the FDA for use for diagnoses using images," Neal reported. "That's definitely an emerging space."

You can tell by their word choice, or the expressions on their face, what level of pain they're in.

FDA-approved systems include AI-powered tools to analyze CT scan images for signs of coronary artery calcification, strokes, brain hemorrhages and potential liver or lung cancer as well as X-rays to detect bone fractures.

In addition, an FDA-approved device analyzes retinal scans to detect diabetic retinopathy. "The first AI solution that does not require physical input," Neal said, "but in general, most machine learning tools that are used for clinical applications still require physician interpretation and interaction by humans."

OCR gains wider recognition

Optical character recognition was among the earliest applications for computer vision tools, but there's a newer way to use AI for document processing -- viewing a picture of an entire document to determine if it requires special handling. Medical images, fingerprints and retinal scans, for example, are sensitive data that require extra protection and are not to be shared with the outside world. The same applies to images of credit cards, drivers' licenses, passports and tax forms.

Avinash Ramineni

"OCR tools are very sensitive," said Avinash Ramineni, co-founder and CTO at cybersecurity software vendor Kogni. "If the document is a little off-kilter or zoomed out or the quality of the scans isn't up to the mark, the OCR fails in getting the text, and a lot of [data loss prevention] tools are relying on the text that comes out of OCR."

Adding image recognition on top of OCR moves the accuracy level from about 70% to better than 99%; it's "a lot more accurate than legacy tool sets," he said.

There are a lot of models out there that are open sourced, and we expanded those models to sensitive data.

Kogni built the software with the aid of open source libraries. "There are a lot of models out there that are open sourced," Ramineni explained, "and we expanded those models to sensitive data, training it on credit cards, W-9 forms, W-2 forms and so on."

Expect to see computer vision tools paired with emerging technologies as well. Warby Parker's Virtual Try-On face-scan feature, for example, "allows customers to sample [eyeglass] frames from their phone," said Zachary Jarvinen, head of technology strategy for AI and analytics at enterprise information management software maker OpenText. And Sephora's Virtual Artist facial makeup app, he added, "helps shoppers test products from the comfort of their couch."