Toyota outage shows not all backup failures are ransomware
An IT error for Toyota Motor Corporation knocked out production for a day, but the auto giant was quick to note the issue was not due to a cyberattack.
Ransomware attacks have become a major cause of enterprise IT outages, resulting in lost time, data and money. But traditional infrastructure failures still loom large.
Toyota Motor Corporation shuttered more than a dozen manufacturing facilities in Japan for a day late last month after a malfunction in its production order system, according to a press release. Toyota was quick to state the outage was due to technical difficulties and not by a cyber attack, which caused a similar one-day stoppage in 2022 when a supplier was attacked.
Toyota, which declined further comment, said in a prepared statement that an error occurred during server maintenance on Aug. 27, resulting in insufficient disk space for a database system. Backup servers, which were part of the production system, replicated the error and were made inaccessible as well. The system was taken offline Aug. 28 and restored the following day, alongside operations, after "data was moved to systems with a larger capacity."
"These are multi-billion-dollar companies. You'd think they'd have backups all over the place," said Ray Lucchesi, president and founder of Silverton Consulting.
These sorts of common outages have existed for as long as IT has been a profession. But the escalating frequency of cyber attacks can make the public and C-suite assume any downtime is from a hack, said Dave Raffo, an analyst at Futurum Group. The damage a cyber attack can have on an organization legally, financially or reputationally requires effective notification. Companies investing in cyber defenses and reinforcements also shouldn't forget their backup basics, he added.
"When people hear about glitches now, they think it's a cyber attack," Raffo said. "This stuff is not infallible. There are no guarantees. The important thing is how quickly you can recover from it."
Ounce of prevention or pound of pain
Enterprises are becoming more proactive in their reporting of outages, either by honest mistake or malicious actors, due to an increasing number of data privacy laws worldwide.
As an international company, Toyota is subject to international regulations including the European Union's GDPR. While the most recent Toyota announcement wasn't spurred by GDPR specifically, its timing and public posting are legally compelled by laws like GDPR. Enacted in 2018, GDPR requires all businesses that trade within the EU to follow specific data stewardship laws or face significant monetary fines, said Marc Staimer, president and founder of Dragon Slayer Consulting.
"Companies don't like to admit when they're hit with ransomware," Staimer said. "[It's] the No. 1 cause of outage today. That didn't use to be the case."
Last month's outage at Toyota may have been due to an IT error, and companies must do more with backup strategies beyond conventional wisdom like the 3-2-1 backup strategy, Staimer said.
The 3-2-1 strategy suggests ensuring three copies are made of the data to be protected, the copies are stored on two different types of storage media and one copy of the data is sent off site.
This strategy can still expose companies to risk. Ransomware can worm into backups as a payload detonation on premises, in the cloud or in disconnected backups.
"You don't know where you have a good copy of that data," Staimer said. "It's flawed; it's a vulnerability. There's no perfect data protection."
TechTarget's Adam Armstrong contributed to this report.
Tim McCarthy is a journalist from the Merrimack Valley of Massachusetts. He covers cloud and data storage news.
