Using digital twins simulation to reduce risks in industry
Whether it's fighting wildfires or working in a warehouse, people often encounter dangerous working conditions. Researchers are using digital twin simulation to counter those risks.
For firefighters, dealing with dangerous wildfires means digesting information and data and making instantaneous decisions about what they're seeing.
Firefighting is among the most dangerous jobs, and firefighters often make life-or-death decisions. A division within Lockheed Martin's AI Center (LAIC) is trying to use AI to quicken those decisions..
The LAIC is using AI technology to help firefighters evaluate the various environmental factors involved in battling wildfires, such as wind conditions and how hard a fire is burning, so they understand exactly what dangers they're facing in the field.
To do this, the LAIC is using tools for digital simulation in Nvidia's version of the omniverse environment of virtual and augmented reality, Irene Helley, director of artificial intelligence integration at Lockheed Martin, said during a panel discussion on March 23 at AI technology vendor Nvidia's Spring GTC 2022 conference.
How Lockheed Martin uses digital twins
Nvidia Omniverse is the AI hardware and software vendor's platform for 3D design and simulation, aimed at creating digital twin representations of virtual working environments to simulate dangerous working conditions such as wildfires.
Lockheed Martin uses Nvidia Omniverse to model the wildfire and visualize its spread in real time with analytics visualization.
The LAIC is also using machine learning tools within the digital twin and simulated environment to create synthetic machine learning training data -- data that is generated artificially instead of by real-world events.
"In this simulated environment, you could fly an asset through a wildfire and have [machine learning models] create learning materials to create better or improved predictions in the future," Helley said.
For the LAIC, using simulation helps firefighters parse through the chaos they are confronted with at the scene.
"A fire really is currently fought through multiple channels from perception and analysis to situation awareness and then finally decision-making and tasking," Helley said.
Experts discuss ways simulation can help in fighting fires and working in dangerous warehouses.
"By creating this mission manager with AI, we reduce that high-touch communication … where responders and observers are radioing in incident commands and what they observe and those observations in turn are laboriously combined and analyzed for the fire behavior and to determine the best course of action," she continued.
Using the simulation … to train our models helps us make the best decisions based on the information we have. We can get right to the heart of courses of action and planning.
Irene HelleyDirector of AI integration, Lockheed Martin
The AI models inform the team at LAIC about the most vital information regarding the fire.
"Using the simulation ... to train our models helps us make the best decisions based on the information we have. We can get right to the heart of courses of action and planning," Helley added.
Other organizations are also using AI and digital twins to maneuver in dangerous working environments.
Kion Group and Sarcos Robotics
Kion Group, a Germany-based manufacturer of materials handling equipment such as forklifts and warehouse automation tools, is using digital twins and simulation to control sometimes dangerous working conditions at its warehouses.
The pressures of efficiently transporting orders on time and narrow aisles in warehouses and factories make it easy for forklift operators and other workers to get into accidents.
"We have real-time localization for people and forklifts, bringing them together in a digital twin," Bengt Abel, project manager for robotics and AI at Kion Group, said during the panel discussion.
The company tries to simulate what could happen when machines and humans are working together in close environments, and ways to prevent accidents involving machines and humans.
Meanwhile, Sarcos Robotics, based in Salt Lake City, is also considering digital twin simulations and ways AI could mitigate risks in dangerous working conditions, said Ed Tiongson, vice president of product management.
While Sarcos is moving towards complete automation of dangerous tasks, sometimes human employees are essential for working in dangerous environments like mines and construction sites.
Sarcos is using digital snakes, robotic tools that crawl through dangerous environments such as rock piles, and large robots that replace humans in those dangerous work environments.
In order to simulate what could happen in a dangerous situation, the Sarcos team has to perform a lot of recognition and processing after robots have performed their jobs at the worksite.
This means having the robot create a digital image that is simulated to determine different reactions that could take place when the robot encounters different options later.
"For us, that's a lot more forward-looking," Tiongson said.
