To prepare for the 2010 Winter Olympics, the Vancouver Canucks used Veeam to protect against point-of-sale system failures -- a major upgrade over its previous tape backup.
As host of the ice hockey events at the 2010 Winter Olympics, Aquilini Investment Group, owner of Rogers Arena and the Vancouver Canucks, had to rethink its entire IT game plan.
Rogers Arena has a capacity of around 18,000 people, and its IT infrastructure had to ensure all ticket scanners, Wi-Fi and point-of-sale systems would never go down during the heavy influx of attendees. In 2010, Aquilini revamped its legacy systems, moving away from physical servers and tape to virtualization and VM backup. It deployed VMware and Veeam backup.
"We were starting to see the serious benefits of virtualization compared to traditional physical servers," said Olly Prince, manager of infrastructure at Canucks Sports & Entertainment and Aquilini Group.
The switch dramatically changed how the Canucks handled backup. Prince described the old system as "hit-or-miss." Backup copies of data were stored on tapes that were then sent to an off-site facility. When a user needed something restored, the correct tape had to be found and then delivered back to the data center for restoration. The whole process took four or five business days, and there was no guarantee that the restoration would succeed.
With Veeam backup, Prince said, he's now able to restore data in 10 minutes.
Cloud considerations hinge on cost
As part of the IT revamp, Aquilini has been looking at the cloud more closely, but has only dipped a toe in. So far, there is a single test/dev workload deployed on AWS that isn't being backed up because of how inconsequential it is. Prince had conducted a cost analysis and found that it's still cheaper to run most workloads in VMs on premises.
Olly Prince
However, Aquilini wants to dive deeper into cloud. Some of the ways the company wants to take advantage of the cloud are disaster recovery (DR), Office 365 backup and to give coaches a way to upload videos or access useful player metadata while they are on the road. Right now, the last option is being achieved by having the team carry a "travel server" with them wherever they go.
"We're looking at everything as a whole and strategizing what makes sense for our organization to do on cloud or on prem," said Margaret Pawlak, IT business strategy and project manager at Aquilini Group, Canucks Sports & Entertainment.
Margaret Pawlak
Aquilini recently finished a proof of concept with Microsoft Azure for DR. Prince said he was able to replicate on-premises applications and run them on the cloud, but the next step is factoring in costs. The company's current DR plan involves replicating and failing over to an off-site facility about 60 kilometers away from the main data center. That site also houses its own separate production environment, so while it has enough storage to bring enough VMs back online to keep the business running, it won't include absolutely everything.
Although Pawlak and Prince said they're actively working on pushing some of these cloud strategies, they're having difficulty convincing the rest of the organization that changes are necessary.
Horror stories don't get you a [cloud backup] budget.
Olly PrinceManager of infrastructure, Canucks Sports & Entertainment
In the case of Office 365 backup, there is a pervasive myth that its native long-term retention policy is a suitable replacement for true, point-in-time backup. Prince pointed out that retention doesn't help when trying to restore a corrupted or deleted file.
In the case of DR, Pawlak said it is hard to put a business case forward for what is essentially insurance. The benefit is not something tangible until a real disaster hits, and there's a belief that such an event will never actually happen. Prince said it's a difficult attitude to overcome until it's too late -- no matter how many times he's shared IT horror stories from his peers in the industry.
"Horror stories don't get you a budget," Prince said.
Backup strategies beyond the rink
Prince's team of four IT personnel, himself included, is responsible not just for the Canucks franchise and Rogers Arena, but for hotels, wineries and other properties owned by Aquilini Group. A total of 180 TB from 60 VMware VMs are being protected by Veeam backup. Aside from the daily business data generated by Rogers Arena, some of the VMs also house audio and visual data, as well as player performance metadata that the Canucks franchise uses for scouting, training and coaching.
Aquilini uses Darktrace for cyberdefense, but Prince focuses much of his attention on user training as well. He said ransomware is more likely to get through unaware staff than through vulnerabilities in devices or workstations they use, so he trains them on how to spot phishing and avoid executing programs they're unsure of. A good backup system is also an important part of the overall security package.
Aquilini would not comment on other data protection vendors that were considered besides Veeam, but Prince said ease of deployment and use were huge factors in the decision, given how small his IT staff is.
Prince said he wants Veeam to work natively with Azure cold storage, which it currently doesn't. On top of certain files that need to be retained for compliance reasons, the Canucks franchise has a large amount of audio and visual files that need to be archived for potential future use. Not all the footage is mission-critical, but some clips might be useful for pulling together a promotional video.
"It would be nice to take a backup of that and shove it somewhere cheap," Prince said.
Olly Prince
Margaret Pawlak
