Behavioral eManipulation: Attacking the care delivery workflow to deliver harm

Attack scenarios

Methodologies
Behavioral eManipulation is best understood in the context of conventional cyberattack methods:

• Technical exploits attack systems. This is where an attacker finds and exploits a security vulnerability (or vulnerabilities) in an application or network. A well-known example of this is when an American teenager hacked the International Space Station’s life support systems, causing $41,000 to repair the damage.
• Social engineering attacks people. This is where an attacker takes advantage of human susceptibility to exploit the human. Perhaps the most famous example of all time is the original Trojan Horse, where a seemingly defeated Greek army left a trophy for the seemingly triumphant Trojans, who then unwittingly brought the trophy inside the walls only to later find it filled with invading Greeks.
• Behavioral eManipulation attacks workflows. This is where an attacker exploits a device or system that an unwitting human victim relies upon to make decisions, and then the workflow delivers the adverse outcome as a result of reliance on that bad information. Examples of this include the variety of scenarios described below in this section.

Behavioral eManipulation has a few discrete phases to the attack flow:

1. Understand the workflow
2. Manipulate the data
3. Rely on humans to unwittingly deliver the adverse outcome
   1. In the context of healthcare, phase 1 is actually a mitigating factor, it requires many years of medical training for aspiring physicians to learn the many care delivery workflows. However, while that limits the potential pool of viable attackers who can use this attack method efficiently, it does not prevent any attacker from acquiring this knowledge.
   2. Phase 2 is where most security research to date has focused.
   3. Not including accidental exploits, phase 3 is only possible when the combination of phases 1 and 2 have concluded.

Diagnostic attack scenarios: Overview
Most medical errors are communication errors. Behavioral eManipulation is essentially an amplification of communication errors, thereby compounding medical errors. In a healthcare context, Behavioral eManipulation manifests itself most dramatically in diagnostic workflows, which are the parts of the care delivery workflow that require a physician, nurse or other medical professional to interpret data in order to diagnose a patient’s health issue, for purposes of prescribing a treatment plan. Such diagnostic scenarios largely fall into one of two categories:

1. Workflows that have failsafes built in, also known as “system of checks,” or
2. Workflows that do not have failsafes and rely heavily on the physician to interpret the data.

Each of these diagnostic attack scenarios has different implications to patient safety if the information upon which the physician bases decisions is manipulated.

Diagnostic attack scenarios: Systems of checks
Diagnostic attack scenarios with systems of safety checks are typically those which involve biographical data, systems or other information which are known to deliver harm if the workflow is flawed. An ever-growing body of statistics demonstrates that errors related to blood and allergies are known to have harmful or even fatal implications; for that reason, healthcare providers have numerous safety checks in place to ensure such errors are minimized. If, in a typical care workflow, the physician notes an anomaly, there are procedures in place that would trigger certain actions¹ by the physician to verify the anomaly before prescribing a potentially harmful procedure or dosage. Through Behavioral eManipulation, an attacker would trigger anomalies that require physicians to re-perform diagnostics, order new tests, and/or consult additional medical professionals. Time is critical in the kinds of settings where such diagnostics would take place (e.g., acute care settings, operating rooms, etc.), and such additional steps all require critical time. As a result, in some cases time impedes care, while in other cases the physician may lose trust in the diagnostics system – assuming it to be faulty – and revert to other diagnostic methods that circumvent the established system of checks. All of these conditions dramatically increase the likelihood of medical error that could result from such manipulation of the care delivery workflow.

Proper workflow

1. Patient produces data
2. Physician reviews data
3. Physician consults duplicative sources for data (failsafe)
4. Physician makes diagnosis
5. Physician administers treatment plan

Manipulated workflow

1. Patient produces data
2. Physician reviews data
3. Attacker exploits security vulnerability*
4. Attacker manipulates duplicative data sources
5. Physician consults duplicative data sources (failsafe)
6. Physician notes anomalies
7. Physician measures patient again
8. Physician reviews new data
9. Physician consults/compares with duplicative sources for data (failsafe)
10. Physician notes anomalies
11. Physician measures patient again
12. Physician reviews new data
13. Physician proceeds on bad data, or loses trust in system
14. Physician makes diagnosis
15. Physician administers invalid treatment plan

*Example technical exploit: As outlined in “Hacking Hospitals,” we discovered a privilege escalation security vulnerability in lobby check-in kiosks that allowed an attacker to take the kiosk out of kiosk mode, gain administrative control, pivot to other systems on the same subnet and manipulate the data accessible by those other systems. One such system was a bloodwork system.

Diagnostic attack scenarios: Data interpretation
By contrast, some attack scenarios do not have safety checks to them and rely solely on the physician’s experience and interpretation of data to process the diagnosis and resulting treatment plan. Examples of these would pertain to monitoring equipment or lab results. The physician is unlikely to question the validity of the information coming off such systems unless the readouts fall unusually outside of expected parameters. As there are not additional failsafes in place, if acting upon manipulated information, the physician operating in the care delivery workflow would likely unwittingly deliver harm.

Proper workflow

1. Patient produces data
2. Physician reviews data
3. Physician makes diagnosis
4. Physician administers treatment plan

Manipulated workflow

1. Patient produces data
2. Attacker exploits security vulnerability*
3. Physician reviews data
4. Physician makes diagnosis
5. Physician administers invalid treatment plan

*Example technical exploit: As outlined in “Hacking Hospitals,” we discovered an authentication bypass vulnerability that, when combined with a remote code execution exploit, allowed a remote attacker to take administrative control over a patient bedside monitor. This enabled the attacker to manipulate the data feeding the screen, thereby triggering invalid physician response or preventing valid and necessary physician response.

Remedies

The safety implications introduced with the attack concept of Behavioral eManipulation are significant; they usher in a new area of how the security community and the device manufacturing community should think about security vulnerabilities. However, many of the recommended solutions to the situation are either long-proven approaches or are new takes on long-proven approaches.

Return on investment: Training physicians versus securing devices
Many security professionals advocate for training physicians as a necessary solution to the many security vulnerabilities that plague healthcare. However, training of physicians is unlikely to lead to a considerable improvement on security posture. This is due to the fact that physician behavior is a byproduct of decades of medical training that prioritizes delivery of care in the most time-efficient manner. The most important aspect of this is that physicians are trained to recognize patterns; when they notice irregularities is when they investigate further for possible intervention. Compound this with the fact that medical professionals already invest extreme hours per week on the core medical training required to become a licensed and effective physician and there is simply insufficient time to teach a new behavior on something that is not aligned with either pattern recognition or the core medical foundation of their education. Given the confluence of these reasons, it is our position that the most effective model for addressing the workflow-related security implications introduced by this article is by focusing not on physician training, but rather on securing the devices themselves. By eliminating the workflow problems at the source of the data manipulation, the entire concept of Behavioral eManipulation is thwarted. What follows are some effective approaches for securing medical devices.

Think like an attacker thinks
First and foremost, it should go without saying that if devices have the ability to be manipulated to arrive at an outcome of harm, then they should first be investigated for how they will be attacked. This is commonly known as a security assessment, or in some vernacular as penetration tests. However, these approaches are significantly different, do not share the same goal and do not both best remediate security risk. Vendors of connected medical devices – especially of passive medical devices, which until now have not historically been considered relevant to patient safety in a security context – should not be satisfied with cursory approaches to security like automated scanning, commodity penetration testing, baseline compliance and deferring risk to the hospital. Instead, vendors should:

• Apply the same level of rigor to security assessment as a sophisticated adversary would;
• Engage in white box security assessments, not just black or gray box;
• Investigate for not just known and common vulnerabilities, but also custom and zero-day vulnerabilities; and
• Work with proven, research-focused, services-only consulting firms.

Security models: Threat model + trust model
Over the years of executing security assessments and performing security research, we’ve found that many companies either aren’t familiar with the concepts of both threat modeling and trust modeling, or of those who do, very few have both implemented. This inherently undermines a successful security mission, as both are required in order to deploy a successful security program. A threat model is an exercise through which an organization identifies what it wants to protect (i.e., its assets), who is it trying to defend against (i.e., its adversaries), and the areas against which attacks will be launched (i.e., its attack surfaces). By contrast, a trust model is an exercise through which an organization defines who and what it trusts, why it trusts those people and systems, and how access based on that trust is provisioned, revoked and validated for authorization and authentication. All device vendors should:

• Define and implement their threat model
• Define and implement their trust model
• Update both frequently
• Work with trusted third-party security experts to investigate how well the actual design and implementation of the systems adheres to both security models

Recognize role of compliance
The Health Insurance Portability and Accountability Act (HIPAA) governs much of what the healthcare industry does related to security. However, many of the issues inherent with Behavioral eManipulation are actually out of HIPAA’s scope. When it comes to medical devices in particular, the Food and Drug Administration (FDA) has published guidelines that outline security approaches relevant to medical device security. However, there is common and widespread misunderstanding about these guidelines and whether they require new rounds of FDA approval – and these approvals typically take many years. In some cases, these guidelines are not enforceable (an issue that this author investigated in a separate analysis).

In either the case of HIPAA or FDA, it is important to note that compliance typically does an adequate job of establishing the baseline requirements for the foundation of a security program; however, compliance should not be seen as the entire security program unto itself. Effective security organizations recognize the role of compliance as being important to satisfying stakeholder needs, but will go beyond the outlined minimum if delivering a robust security posture is important. To be effective in this domain, organizations should:

• Define what a successful outcome of the security model looks like,
• Define the delta between compliance and the desired outcome, and
• Mobilize security investments accordingly to address the delta between regulatory compliance and security mission.

Summary

Behavioral eManipulation is the combination of technical exploit and human behavior to arrive at adverse outcomes. It represents an evolution in long-standing attack models, and should be immediately incorporated into defense schema across all organizations in all industries, with the baseline condition that those organizations have something worth protecting. For the purposes of this article, we narrowed the scope of the analysis to focus on patient harm issues related to exploited passive medical devices, but the principles of attacking workflows are relevant across virtually every industry, such as the Stuxnet example provided earlier. We advocate for suppliers of applications, devices and systems to work with security experts to identify and remediate any workflow-related security risks that may not have historically been considered in threat models and trust models for your systems. This research will first be presented at RSA Conference USA 2018.

Konstantinos M. Triantafillou, M.D. is a practicing orthopaedic trauma surgeon, certified by the American Board of Orthopaedic Surgery. He serves patients at University of Tennessee Medical Center, a Level 1 trauma center. Dr. Triantafillous is active in advancing the field of orthopaedic surgery through research, and pursues his interest in medical security due to concerns from a practitioner’s perspective about the vulnerability in medical devices deployed across surgical, acute care and in-patient settings. He is widely published across medical journals and has presented medical research at conferences such as the American Academy of Orthopaedic Surgeons. He pursued his BA, MD and residency at Georgetown University, and his fellowship in orthopaedic traumatology at the renowned Campbell Clinic.

¹ For the purposes of this article, we intentionally leave details of the care workflows opaque, as it would be irresponsible to equip malicious entities with an attack blueprint on well-established medical workflows that are unlikely to change quickly. Such details can be disclosed under signed agreements – contact authors for more information.

All IoT Agenda network contributors are responsible for the content and accuracy of their posts. Opinions are of the writers and do not necessarily convey the thoughts of IoT Agenda.