Backup retention policy best practices: A guide with template

What is a backup retention policy?

A backup retention policy is an internal organizational rule that determines what data the organization keeps, where it keeps the data and how long it keeps the data.

Click here to download
our free backup retention
policy template.

A retention policy might indicate the types of backup that are acceptable. For example, it might stipulate that the data must be on multiple backup media, such as tape or cloud. Different methods, such as full backups, incremental backups and differential backups, might also be part of a retention policy.

Retention policies exist for numerous reasons, and they often ensure that customer or client data is secure and accessible. Industries such as healthcare, education, IT and retail will all have different requirements in their respective retention policies. Some retention policies might also include rules for data that must be deleted after a certain period.

Why do organizations need backup retention policies?

Backup retention policies specify how long a backup must be kept after it has been created. These policies are important for several reasons.

One reason why backup retention policies are so important is that many businesses are subject to compliance mandates that govern backup retention. Adhering to a backup retention policy is an essential part of maintaining the organization's regulatory compliance.

Another reason why backup retention policies are important is that such policies can be driven by business requirements. A business might, for example, determine that in the event of a major data loss event that also affects recent backups, a six-week-old backup would still be of use, but older backups would not. Such a business would need to create a backup retention policy establishing that all backups must be retained until they are more than six weeks old. This type of policy would prevent older but potentially useful backups from being purged.

Backup retention policies also affect storage costs, since there is a direct expense associated with creating and retaining a backup. This is especially true when backups are being written to cloud storage or immutable storage, both of which are usually billed on a per-gigabyte, per-month basis. While a backup retention policy's primary purpose is to specify how long backups must be kept, it also provides guidance regarding which backups can be deleted. Deleting older backups removes the costs associated with storing those backups.

Retention policy considerations

When creating a retention policy, three major considerations must be made: what data the business must retain, any relevant compliance regulations and future policy updates.

What data to retain

Some data is required by law to be retained for a certain time frame; other data is nice to keep around, but isn't legally required by a retention policy. Organizations can also implement internal rules around what data they retain and for how long, assuming they meet or exceed compliance regulations. Common types of retained data include files, email messages and database records.

One of the first backup retention best practices to keep in mind is knowing what data should remain live and what data the organization should archive. Typically, this determination is based on data age, but that is not always the case. In some cases, admins examine criteria such as when the data was last accessed and the data type.

For example, an organization might have plenty of free space on the file server, but want to cut down on some of the clutter. With this data deletion goal in mind, it decides to create an archive policy that moves anything older than five years to the archives, and then deletes anything more than 10 years old.

Although this might sound like a reasonable approach to creating a data retention policy, it could have unwanted consequences. What happens if a spreadsheet was created six years ago, but is regularly updated? If the data retention policy only looks at the creation date, then the spreadsheet would be archived, even though it is regularly used. It tends to be much more effective to base a retention policy on the last access date rather than the creation date.

Data retention policies can also backfire in other ways. For example, if an organization signed a 15-year lease for its office building 11 years ago, in all likelihood, nobody has looked at the document in the last 10 years. However, the organization probably wants to keep it. The policy should consider instances of this nature.

Compliance

One major reason that a company retains data is compliance. In addition to its internal compliance rules, a company needs to consider several laws and regulations in forming its data retention policy. Figuring out the applicable laws is important; an outside auditor can help.

The European Union's GDPR, for example, which went into effect in May 2018, features mandates that apply to personal data produced by EU residents, no matter where it's stored. A data-collecting organization should have a data retention policy that specifically outlines GDPR compliance issues.

Other regulations that feature data retention requirements include the Sarbanes-Oxley Act in the U.S. and the Payment Card Industry Data Security Standard. Especially as it relates to these regulations, an organization should only keep personal data that's needed.

Regulatory compliance is a common business concern. Penalties for violations include fines and loss of reputation. A data retention schedule within an internal policy can be a helpful tool for compliance. Organizations must also ensure that policies are updated to reflect changes in data production and compliance or data security laws.

Policy updates

An organization must be able to dynamically alter its retention policies if necessary. For example, an organization subject to litigation might be required to retain all backups from a certain time period, even if the organization's backup retention policy would ordinarily allow the backups to be deleted. Policies must be adaptable and capable of being updated as needed.

Backup retention policy and scheduling checklist

Nuances in retention policies will vary by organization, but the checklist below outlines some basic, necessary steps to develop a solid backup retention plan. Admins can use the following schedule when they create a data retention policy:

1. Define the data.
2. Organize the data by lifecycle.
3. Determine the number of versions to store.
4. Outline backup type and frequency.
5. Create a lifecycle policy for each data set.
6. Delete and purge unnecessary files.
7. Review and run the backup retention policy.

The final step is crucial to a successful policy, so don't skip out on the review process. This step will let admins know if there is anything that they must update, ideally before it affects client data.

Best practices for backup retention policies

Below are some backup retention best practices that admins can reference when they create a new policy for their organization. Some regulations and security concerns might not affect certain organizations, but general best practices for backup admins include the following:

• Consider how industry regulations will affect the retention strategy.
• Always consider backup data sets, type and frequency.
• Identify and address restoration scenarios.
• Keep incremental backups within a reasonable size.
• Keep the last backup in an easily accessible location.
• Evaluate cycle-based vs. time-based backup retention.
• Ensure that there is enough storage for data backups.
• Schedule backups for when the organization has the most available bandwidth.

Where to store backups

Rules and regulations often determine a retention period. Since retention periods range from minutes to years, an organization might need different types of media for storing data.

Some types of storage media, for example, have a 15-year data integrity guarantee. After that time, the data might begin to degrade. So, if an organization plans on keeping certain data for 50 years, does it make sense to store that data on a device that is only rated for 15 years? Depending on the data's longevity, a backup retention policy might need to address the required media types or even set up a plan for rotating aging media.

The public cloud is a popular storage location for long-term retention. Amazon S3 Glacier, Microsoft Azure Blob Storage and Google Nearline are among the options for low-cost archival storage in the cloud. The storage is off-site, which is good for data protection. Restore times and costs can run high, though, depending on how much an organization needs to bring out of the cloud.

Tape is another media type for long-term storage that is cheaper than other options, such as disk. Durability is typically stated at up to 30 years for the latest LTO tape cartridges. LTO-9 provides 45 TB of compressed storage capacity. Restore speed is slow, however, so an organization shouldn't solely use tape to retain data that needs quick recovery.

Disk is more expensive, but faster than tape. It's not cost-effective for storing large amounts of data that need long-term retention and that the organization probably won't access frequently.

A solid backup retention policy might use the 3-2-1 backup method. This method might be a bit simple given today's variety of backup options, but it is a good place to start for admins outlining a policy.

How to implement a backup retention policy

There are several steps involved in implementing a backup retention policy:

1. Identify the business and compliance requirements for your data. Remember, different data sets might have varying retention requirements.
2. Establish various retention tiers -- such as short, medium and long -- and define a period of time for each tier. You can then begin mapping your various data sets to the appropriate retention tiers.
3. Determine your backup frequency. The frequency with which you back up your data will determine the recovery point objective for that data. In other words, the more frequently you back up your data, the less data could potentially be lost because of an adverse event. Some organizations tie the backup frequency to the data's change rate -- for example, some data is backed up every few seconds, while other data might only be backed up once per day.
4. Choose your preferred storage media and location. As a best practice, create multi-tier backups in which data is stored in multiple locations and on at least two different types of media. When choosing your backup media types, consider the media's estimated lifespan, since the media needs to remain viable for long enough to satisfy your data retention requirements.
5. Define your backup security and access controls. Part of the data retention policy creation process is determining how you will secure your backups and how you will control access to these backups. In some cases, the security and access controls will be governed by regulatory requirements. In other cases, it will be up to the organization's own discretion. As you evaluate your security and access control options, it's important to balance security with accessibility. You never want to apply so much security to a backup that you are unable to access the backup in a time of crisis because dependency systems are offline.
6. Automate policy enforcement. Backup retention policies need to be automated to be effective. As such, you will need to determine what controls you will put in place to ensure that backups are kept for the required amount of time and automatically purged when the retention period expires.
7. Test and validate your backup retention policy. You can't assume that your backup retention policy is going to work as intended. You need to establish some procedures for testing your automated policy enforcement to ensure that it is working properly.
8. Document your policy and train the IT staff. Your backup retention policy needs to be formally documented. You will also need to explain the policy and its control to the IT staff to ensure that policies are not violated or circumvented.
9. Regularly review and update the policy. The backup retention policy that you put into place today might not meet your needs a year from now. It's essential to schedule regular reviews of your backup policy and to have a system in place for adapting the policy to changing business needs or regulatory requirements.

Don't forget to check out the included backup retention policy template for an example.

Brien Posey is a former 22-time Microsoft MVP and a commercial astronaut candidate. In his more than 30 years in IT, he has served as a lead network engineer for the U.S. Department of Defense and a network administrator for some of the largest insurance companies in America.