Qumulo File Fabric fuses on-prem, cloud storage for studio

For years, visual effects studio FuseFX used Amazon Web Services for cloud bursting for compute while keeping most of its storage on premises. FuseFX had several storage platforms, but none gave it the ability to scale without limits in the cloud, according CTO Jason Fotter.

Fotter finally found his AWS storage answer when Qumulo launched its File Fabric last September. Qumulo File Fabric (QF2) runs on premises and in AWS, allowing customers to replicate data between them to set up globally distributed storage clusters.

FuseFX has 300 employees in offices in Los Angeles, New York and Vancouver. The studio mainly focuses on television shows such as The Tick, The Walking Dead and The Blacklist, and also provides effects for movies.

Fotter said his in-house storage includes Qumulo, Dell EMC Isilon, Quantum StorNext and a Rozo Systems array for backup.

An Isilon cluster serves as FuseFX's main production storage. Qumulo serves as storage for the studio's Houdini Simulation software, StorNext enables real-time 4K video playback and Rozo protects it all.

"We use different storage systems for different workloads," Fotter said. "I've been telling my storage vendors it would be great to have a cloud storage product -- a clustered file system that sits underneath our render farm and supports any size workloads. Qumulo was the first to come out with a product that hit the marks we needed and worked. It's an elastic workflow that is on demand."

Isilon CloudPools support public clouds, but some features rely on Dell EMC's Virtustream or Elastic Cloud Storage. And CloudPools only support Isilon storage. Fotter said QF2 lets him synchronize any on-premises storage to a Qumulo cluster in AWS.

"It's been a challenge for them to present me with a product that makes sense for us," Fotter said of Dell EMC. "Our whole workload is on AWS. We won't go down that road of using a private cloud."

Our whole workload is on AWS. We won't go down that road of using a private cloud.

FuseFX already had a four-node Qumulo QC208 cluster with about 450 TB on premises. After adding Qumulo File Fabric, it spun up another four-node cluster in AWS to burst during busy periods.

FuseFX uses the cloud mainly to generate final images, a process known as rendering. AWS gives FuseFX elasticity -- the studio works on multiple projects simultaneously, usually on tight deadlines.

FuseFX used Bracket Computing software since 2014 to spin up and manage resources in AWS, along with a home-grown Linux-based file storage system in the cloud. But that storage could not scale to the large workloads FuseFX requires. At peak, the studio requires a thousand AWS EC2 Spot Instances and the Linux system struggled to keep up with more than 400, Fotter said.

Fotter said last year Bracket switched its focus to cloud security, and FuseFX needed a new process for working in AWS.

"We've been using the cloud a few years now for rendering," Fotter said. "It's a bust scenario. We have local render farms in each office, but there are times when we have more jobs to render than capacity. We developed a cloud workflow where we can synchronize our assets to cloud storage -- which is now Qumulo. Then we spin up and virtualize as many Amazon Spot Instances as we need."

Qumulo File Fabric uses a custom Amazon Machine Image to spin up virtual Qumulo nodes. FuseFX spins up four-node clusters with Amazon Elastic Block Storage volumes attached. The clusters provide FuseFX with solid-state drive and hard disk drive storage in AWS.

"We put it on the network, configure it to give it a DNS entry, and we're off and running," Fotter said.

One four-node Qumulo File Fabric cluster on AWS supports 1,000 32 Core AWS EC2 instances, Fotter said. He said the peak performance was 40,000 IOPS with 3.87 GB per second throughput.

FuseFX's proprietary Nucleus production management system sits between on-premises and cloud clusters. Nucleus synchronizes the files, serving them up to users at one of FuseFX's three offices. "So the cloud becomes a virtualized fourth location for us," Fotter said.