How did a Microsoft Equation Editor flaw put systems at risk?

Microsoft Equation Editor had a stack buffer overflow vulnerability that put systems with the program on it at risk. How did the vulnerability work, and what fixes are available?

Microsoft Equation Editor, a component of Microsoft Office, is an out-of-process component object model server, and it is an executable file named eqnedt32.exe.

The vulnerability enables an attacker to execute code remotely when a victim opens an affected RTF document in Microsoft Word. Targeting the Equation Editor enables attackers to bypass system defenses put in place to protect Microsoft Office because Equation Editor is invoked not through Office, but rather by the Windows DCOM Server Process Launcher service.

Both the Enhanced Mitigation Experience Toolkit (EMET) and Windows Defender Exploit Guard -- the replacement for EMET in Windows 10 -- were unable to protect against the vulnerability. In particular, the attack surface reduction feature of Windows Defense Exploit Guard failed to protect against this type of attack.

Without the /DYNAMICBASE flag set, the Equation Editor executable file, eqnedt32.exe, can be reloaded to a non-randomized location. When the flag has been set, it turns on the address space layout randomization (ASLR) feature in Windows.

Windows 7 users are protected from the buffer overflow vulnerability as long as EMET is configured to always use ASLR at a system-wide level. Later Windows versions enable bottom-up ASLR to be set at the system level without the flag, but it is not possible for EMET or Windows Defender Exploit Guard to configure bottom-up ASLR at a system-wide level.

The most convenient fix is to apply an update for the Microsoft Office memory corruption vulnerability, as addressed in CVE-2017-11882. If the update is not available, the administrator can add EMET or Windows Defender Exploit Guard protections to eqnedt32.exe.

System-wide ASLR in Windows 8 and later must be enabled to block the code reuse attack, as described by the CERT division of the Software Engineering Institute at Carnegie Mellon.

If the Microsoft Equation Editor is used infrequently, it can be disabled by importing the registry values as described by CERT in its Vulnerability Note on the issue. The values can be reset if necessary to enable the Microsoft Equation Editor at a later date.

Ask the expert:
Want to ask Judith Myerson a question about security? Submit your question now via email. (All questions are anonymous.)