As the WORM turns: The pros and cons of WORM disk vs. tape

More on data archiving Choosing a data archiving strategy: Disk archiving vs. tape archiving Data backup vs. data archiving: Is data backup closing the gap? Data archiving reduces data backup workload prior to data deduplication

However, the motivation for using WORM hasn't changed much -- it's just the thing for data that needs to be preserved unchanged, for a long time. One of the principal drivers has been government regulations, especially SEC rules 17a-3 and 17a-4, which require broker-dealers to create and store "in an easily accessible manner," a full record of each securities transaction they make in a non-rewriteable and non-erasable format. The rules, originally promulgated in 1997, have been interpreted by the government to include not only inherently non-erasable media but also media provided with equivalent protection through other means, thus the open door to software and hardware protection schemes that allow speedy, standard disk drives to acquire WORM attributes. However, much of the burden of achieving that protection is left to the individual. Even those not specifically constrained by regulation as to the type of media they must use, find that the WORM concept offers security in the form of assurance that best efforts have been made to maintain and protect important or sensitive data.

David G. Hill, principal at Mesabi Group LLC, an analyst and consulting firm, said that the write and erase protection used with magnetic disk comes from software at the operating system level of a storage system, either at the network-attached-storage (NAS) "head," or at the storage controller/server level. To accomplish this "trick" -- making an inherently re-writable media act like it is not -- is usually the result of vendors developing their own proprietary, often Linux-based, operating system. So equipped, not even an administrator can make changes to a WORM disk. Of course, he noted, physical protection schemes (tamper-proof packaging and enclosures) are still important because some WORM disk drives could be susceptible to erasure or rewriting if they were moved into a different system. So, when you are considering WORM storage, be sure to understand how the vendor ensures against device tampering.

The advantages of WORM disk

Hill said one of the advantages that WORM disk possesses relative to WORM tape is that, notwithstanding its "protection" feature, organizations can still easily delete expired data, assuming this capability is enabled. That means organizations can potentially reuse the WORM storage capability indefinitely. However, he said, WORM does present a planning dilemma, namely that a sudden increase in data needing WORM retention might exceed the capacity of the storage array. In a tape system, it's a relatively easy matter to acquire and write data to new cartridges. The matter requires more planning and potentially more money when the WORM solution is disk based.

For those who aren't required by rules 17a-3 and 17a-4 to implement WORM storage, the technology still represents an appealing option. Brian Babineau, an analyst at Enterprise Strategy Group, said the "pro" of WORM is that you are ensured that the data is going to be around as long as you want it to be around and there is very little risk of corruption. If you need to prove the authenticity and integrity of data, WORM can provide that. "The cons are that once you put it on disk it is there until the retention policy expires. So, WORM should be selectively applied rather than broadly applied," he said.

For example, said Babineau, if you do not have a specific data retention policy you might not want to use WORM because you may inadvertently retain data longer than you really need to. Thus, he said, if you are looking at data that is just part of your BI process, "you may want to keep it on a more mutable type of storage."

Another potential complication of WORM disk is that there can be "nuances and subtle configuration changes that are needed to put data into WORM on disk," he added. Likewise, vendors may vary in how they implement WORM controls such as administrative rights and physical device security. "Each vendor has a unique approach," he said.

But on the whole, Babineau said WORM disk is fairly easy to implement and there are many vendor options. For instance, he mentions EMC Corp.'s Centera, which uses an API to generate a secure 128-bit code connected with each unit of stored information. The metadata for the data object is then stored in an XML file called a C-Clip Descriptor File (CDF), which includes information on where to locate the data. Two copies of the CDF and the "object" are stored and future access to the data requires that the application submit the CDF to Centera via the API. This whole process eliminates traditional file names and directory structures. Data "objects" cannot be deleted prior to the end of a predefined retention period.

There is more data storage on tape now than at any time in history. Greg Schulz founder and senior analystStorageIO Group

Babineau said almost every major storage vendor now offers WORM disk as an option. Although he added that he does not track magnetic disk WORM pricing closely, he said they are typically priced similarly to ATA storage -- "in the $2,000 to $5,000 per terabyte range."

Greg Schulz, principal and analyst at StorageIO Group, said you should still expect to pay "a higher than entry-level" price compared to typical iSCSI or Fibre Channel (FC) drives. He pointed out that one of the benefits of new disk-based WORM is that some systems --again, he cited EMC Centera -- support what is sometimes called content-based archiving, which means the WORM capability can be on any volume that you want, so that the array as a whole remains general purpose. "WORM capability is there if you want to use it but it isn't all-or-nothing," he said.

Is WORM tape dead?

Does all the new functionality available in disk mean that WORM tape is dead? No, said Schulz. "There is more data storage on tape now than at any time in history," he noted. What is changing is that disk is helping to keep tape around longer because people are using disk more in a staging and caching role. Choosing to use WORM in either medium is a matter of understanding your requirements. "You need to determine what you need versus what you want," said Schulz.

Hill at Mesabi advised WORM disk shoppers to select an array that can expand if necessary in the future. Some vendors also use the idea of virtual disks for providing WORM disk functionality. Hill said encryption is the key enabling technology for virtualizing WORM, "because it lets the data be located anywhere and, by having the key expire at a certain date, allows you to get rid of data when it is no longer required." And, he added, virtualization tends to improve disk utilization.

About this author: Alan Earls is a Boston-area freelance writer focused on business and technology, particularly data storage.