Part of:Linux backups for new users, pros and everyone in between
How to use the duplicity backup tool
Open source tool duplicity can help sys admins manage many file servers, as well as back up and restore large amounts of data. Learn how to install, run and troubleshoot duplicity.
Backup utilities play a critical role in effective Linux system administration. With proper use, the duplicity backup tool can become an asset to your backup infrastructure.
Despite the popularity of cloud storage, file servers are still essential to most large organizations. Home and small business users also generate a large number of text documents, image files, sound clips and other essential data.
Many Linux backup tools and file management utilities enable sys admins to handle files effectively, including the following:
Cpio is a file archive utility to help bundle and back up files.
Dump-restore is a general backup utility for entire file systems.
Rsync is a file synchronization tool that only copies changed files.
Tar is for bundling and compressing files for transport using protocols like Secure Copy Protocol (SCP) or FTP.
Deja Dup provides a graphical interface for duplicity.
Kopia, a modern data snapshot tool, includes compression, deduplication and encryption.
Time Machine is a macOS backup utility.
Duplicity has several advantages over these and other similar tools, including encrypted backups to common network and cloud targets. It also has cross-platform support for Linux, macOS and Windows. Sys admins can script it to customize its functionality.
This article explores how duplicity relates to the tools listed, best practices for using it and how to troubleshoot issues. This tip is best for beginner to experienced backup admins and Linux users.
What is duplicity?
Duplicity is a common business backup utility. It is open source and cross-platform, though it's best used on Unix-based systems such as Linux, BSD or macOS. The tool is written in Python.
Some of duplicity's features include the following:
Remote backups over many standard protocols, including Secure Shell (SSH), SFTP, SCP, rsync, Amazon S3 and Google Cloud Storage.
Administrators run duplicity from the command line, enabling them to configure advanced settings and integrate backups with other scripted tasks. The Deja Dup utility is a graphical front end to duplicity, making it easier for less command-line savvy users. Deja Dup maintains many of duplicity's common features while providing a much easier interface for most users.
Deja Dup might come installed on some Linux distributions, such as Ubuntu. You will need to use your distribution's package manager to add duplicity if you want to use it on a system without it installed.
Duplicity is a powerful backup utility for sys admins who manage several file servers with a lot of data.
Like duplicity, Deja Dup can write backup jobs to remote and cloud storage locations.
Using duplicity
Duplicity is a powerful backup utility for sys admins who manage several file servers with a lot of data. The tool works best when users automate it via scripting and schedule it with a systemd timer.
This tutorial demonstrates the necessary steps for managing backups with duplicity, but you will likely integrate these steps into a larger scripted process. For example, you might configure weekly full backups supported by daily incremental backups.
Use the following commands to install duplicity. Type the following command if you use the DNF package manager.
$ sudo dnf install duplicity
Type the following if your distribution uses APT.
$ sudo apt install duplicity
Run a backup
The general duplicity syntax is the duplicity command followed by options, the data source and the backup destination.
duplicity [options] source destination
For example, type the following to back up a directory at /projects to a remote host directory named /backups using scp.
This syntax should be familiar; it is the standard "from here to there" structure of many Linux copy commands.
Use the -dry-run option to simulate a backup job as part of your testing process.
The first backup job is a full backup, meaning it grabs all specified files. Once that is complete, duplicity automatically runs incremental backups of only changed files. These backups are much less time-consuming. Be sure you recognize that the initial full backup can take a very long time, depending on the content you specify. You can force a full backup anytime by adding the full option to the command string.
Administrators can configure additional settings, including GnuPG keys, for managing encrypted backups. You'll configure GnuPG key pairs as a separate task and then call the keys as variables in a script.
Most tutorials focus on backup jobs as the critical tasks, but restoring the correct data quickly is essential. The duplicity restore command syntax is just the reverse of the backup command.
$ sudo duplicity [options] destination source
Use the following command if you need to restore the sample backup job given previously.
Test this capability carefully to ensure you're familiar with the process. Restores are typically the result of a data loss incident, so you don't want any sort of delay that affects users or services. Document the restore process in case other administrators need to implement it.
Troubleshooting tips
One of the most common issues with backups is managing the target location where the backup job will be written. Check this first if your duplicity jobs fail.
Ask the following questions when troubleshooting duplicity issues:
Is the storage disk or space accessible?
Is the storage disk or space full?
Does duplicity have write permissions for the storage space?
Is network congestion preventing backup jobs from being completed?
Is another process interfering with the backup job?
Is the disk I/O able to support the backup job?
Verify any duplicity or Deja Dup configurations as part of your troubleshooting process. Don't forget to use the -dry-run option to simulate a backup.
Damon Garn owns Cogspinner Coaction and provides freelance IT writing and editing services. He has written multiple CompTIA study guides, including the Linux+, Cloud Essentials+ and Server+ guides, and contributes extensively to TechTarget Editorial and CompTIA Blogs.
