Privacy and security risks surrounding Microsoft Recall

How does Microsoft Recall work?

Recall is designed to improve efficiency and enhance productivity by letting users instantly retrieve past content viewed on their devices -- such as webpages, applications, images and documents -- through natural language commands. For example, a user could ask Recall, "Open the PDF I was reading last week about configuring solar inverters," eliminating the need to manually search for the document.

The feature uses local AI models to search and retrieve previously viewed content. Users can select which applications or websites Recall monitors, such as web browsers or PDF readers. Once enabled, Recall captures snapshots of these applications, saving them in a folder on the end user's hard drive, and stores gathered usage data in a SQLite database.

Recall can work without an internet connection and doesn't require logging in to a Microsoft account. It currently records only image snapshots, not audio or continuous video content. Microsoft gives users the options to disable the feature -- although it can't be completely uninstalled -- pause it temporarily and delete stored snapshots.

Initially slated for a June 2024 release, Microsoft postponed Recall's launch until October due to numerous security concerns raised by experts. For instance, ethical hacker Alexander Hagenah developed a command-line tool called TotalRecall that could extract and display data from the Recall database in Windows 11, exposing sensitive information about a PC's activity and previous snapshots.

The October 2024 release is limited to participants in the Windows Insider program, enabling Microsoft to gather user feedback and address potential issues before a larger rollout.

Technical requirements for Microsoft Recall

According to Microsoft, Recall is currently only an option on Copilot+ PCs. These new Windows 11 devices are equipped with neural processing units (NPUs), a specialized type of chip designed to handle AI-heavy tasks like real-time translation and image creation. These specialized processors can execute over 40 trillion operations per second, which is crucial for running complex AI algorithms efficiently.

Recall also has several memory and processing technical requirements. These include 16 GB of either DDR5 or LPDDR5 RAM, eight logical processors, and 256 GB of total storage capacity, with at least 50 GB free for Recall's use. The feature is currently supported in only the following languages: English, simplified Chinese, French, German, Japanese and Spanish.

These requirements might change as Microsoft adjusts the feature based on user feedback and technological advancements. For example, Microsoft is working to support Recall on devices with conventional processors, including AMD and Intel chipsets, not just those powered by NPUs.

Security and privacy risks associated with Microsoft Recall

While the Recall feature might appeal to some user segments, it poses significant privacy risks and security vulnerabilities for the majority of Windows users.

Unencrypted database

Recall stores snapshots of user activity in a SQLite database, which is not encrypted. This lack of encryption makes the data susceptible to malware attacks and unauthorized access, especially if an external party gains physical access to the device.

Inability to uninstall

Microsoft positions Recall as a permanent Windows feature, meaning that it comes preinstalled on Copilot+ devices and cannot be completely uninstalled. Thus, even if users disable Recall, it remains on their device as a deactivated keylogger-like feature -- which could be reactivated by malware or a misconfiguration.

Vulnerability to cyberattacks

As part of Microsoft's AI toolkit, Recall depends on a large language model to understand and process users' natural language queries. However, this reliance on LLMs makes Recall susceptible to the same threats that affect LLMs, such as prompt injection and extraction attacks:

• In a prompt injection attack, a threat actor could gain access to the target device and manipulate the Recall feature by crafting a query that causes the system to reveal sensitive information stored in its database.
• In extraction attacks, threat actors might attempt to extract sensitive data from the Recall database and the folder containing snapshots. Examples include snapshots of sensitive documents, online banking portals, or files containing trade secrets and personal information. This would help hackers formulate a comprehensive picture of a user's computer activity, making it easier to execute additional cyberattacks.

Invasion of end users' privacy

In an enterprise context with heavy workloads, Recall could be useful for simplifying processes such as quickly retrieving a file an employee was working on a month ago. For example, a financial officer could access the exact page in a budget file from the previous week using a simple voice command.

But many Windows users are home users who don't need such a feature -- and might feel uncomfortable with the idea of activities such as gaming, chatting with friends and watching movies being recorded. It's akin to having a keylogger constantly running in the background, which raises concerns about exploitation for malicious purposes.

How has Microsoft responded to these privacy risks?

Following security researchers' criticism of Recall, Microsoft postponed its release and implemented several measures to address security concerns:

• Opt-in activation. Microsoft now lets users choose to activate Recall manually, rather than having it enabled by default.
• Encryption. Recall data and snapshots are now encrypted using Device Encryption or BitLocker, enabled by default on all Windows 11 systems. This ensures that the stored data remains protected, even if an unauthorized person gains physical access to the device.
• User control over data retention. Users can delete snapshots at any time and set a storage limit for Recall. Once the limit is reached, older snapshots are automatically deleted to make space for new ones.
• Selective monitoring. Users can prevent Recall from monitoring specific applications or websites. By default, Recall will also not record the screen when users are browsing the internet in private or incognito mode. This feature is supported on major web browsers, including Microsoft Edge, Mozilla Firefox, Opera, Google Chrome and other Chromium-based browsers.
• Digital rights management safeguards. Recall does not record DRM-protected content, ensuring compliance with copyright laws and preventing the capture of sensitive media content.
• Enhanced authentication. Recall access is protected by Windows Hello enhanced sign-in security. Requiring biometric authentication to access Recall adds an extra layer of security.

What are the broader implications of using Microsoft Recall?

The broader implications of integrating Recall into future versions of Windows span security, privacy, workplace dynamics and technical considerations.

Security threats

Well-funded threat actors, such as those backed by national governments like China, Russia and North Korea, are likely to invest heavily in finding ways to infiltrate Recall, potentially using it as a surveillance tool to steal sensitive data from target computers. This could escalate espionage activities and data breaches on a global scale.

Additionally, organized criminal groups and advanced persistent threat actors are likely to be interested in developing malware that can activate Recall on devices where the feature has been disabled. If successful, this could turn Recall into a powerful surveillance tool for collecting sensitive data, with catastrophic consequences for affected companies.

Privacy implications

The comprehensive data collection inherent to Recall raises significant long-term privacy concerns. As the feature becomes more prevalent, there is likely to be increased scrutiny from privacy advocates and regulatory bodies, such as those responsible for enforcing the GDPR, Payment Card Industry Data Security Standard and HIPAA. In regions with strict data protection laws, such as the European Union, Recall might require modifications to comply with regulations like the GDPR, further complicating adoption.

Workplace impact

Recall could also change the workplace environment in subtler ways. For instance, employees' concerns about surveillance could negatively affect morale and trust in their employers, especially if they fear that their personal activities are being monitored inappropriately on company devices. Moreover, although Recall could boost productivity by making it easier to retrieve previously seen information, overreliance on the technology could hinder critical thinking and problem-solving skills.

Technical considerations

Recall is designed to run on Copilot+ PCs that use Arm processors, which raises potential compatibility concerns. Many companies will need to purchase new hardware to use Recall, leading to increased costs and slower adoption.

Furthermore, many standard business applications are not optimized for Arm architectures, which could cause performance issues or even total incompatibility. This limitation is a significant barrier for businesses that rely on software that doesn't run well -- or at all -- on Arm-based systems.

Suggestions for improving Recall before public launch

In addition to the previously mentioned security updates, Microsoft could consider several other measures to improve Recall's security and privacy prior to the feature's public release.

Content awareness

Recall should be configured to conceal sensitive content in stored snapshots. For example, information stored in snapshots should be blurred when accessing a banking portal or checking medical records. To achieve this, Recall could implement intelligent recognition of sensitive data types -- such as credit card numbers, Social Security numbers, banking account information and medical records -- so that it can automatically blur this information in snapshots.

Enabling this capability would require Recall to recognize the content of captured snapshots. This adds some complexity and poses its own privacy risks, but given that Recall already stores everything it sees on screen, blurring sensitive content would align with Recall's existing functionalities and add privacy to the process.

Enhanced encryption

Although Recall encrypts data at rest, this protection doesn't cover scenarios wherein threat actors gain access to a device while it is running and data is decrypted. To mitigate this risk, Recall data should be encrypted both at rest and in use.

Introducing end-to-end encryption for all Recall data, including during processing, would greatly enhance security. Recall could use trusted execution environments to ensure that data remains encrypted and protected even while in use, reducing the risk of unauthorized access during active sessions.

Application-specific data management

In Recall's current configuration, even if an application is removed, its snapshots remain stored in the Recall database. To improve security, Recall should delete all associated snapshots when a user uninstalls an application such as an email client or web browser, ensuring that no sensitive data is left behind. This feature could also be extended to let users review and selectively delete application-specific snapshots before uninstalling, improving transparency and users' control over their data.

Nihad A. Hassan is an independent cybersecurity consultant, expert in digital forensics and cyber open source intelligence, blogger, and book author. Hassan has been actively researching various areas of information security for more than 15 years and has developed numerous cybersecurity education courses and technical guides.