Getty Images/iStockphoto
DreamWorks Animation taps NetApp in transition to cloud
DreamWorks Animation built a microservices-focused hybrid storage architecture by combining NetApp flash storage and software with Microsoft Azure cloud services.
Ruby Gillman has a growing problem balancing her teenage drama with her ancient birthright as a massive sea monster -- the kraken -- in DreamWorks Animation's new film, Ruby Gillman, Teenage Kraken.
Animators and artists working for the studio in Glendale, Calif., faced growing problems of their own bringing her journey to the silver screen, according to Scott "Skottie" Miller, a vice president and technology fellow who oversees systems architecture at DreamWorks Animation.
The team needed fast, highly available storage to capture the scale of Ruby and the simulation of the ocean surrounding her seaside world.
"[It] put a lot of pressure on our effects simulation, on our modelers and our animators," Miller said.
Miller and his fellow executives report to the DreamWorks CTO, providing advice on new technologies and working alongside creative and tools teams to develop capabilities needed for each feature.
NetApp flash storage systems in DreamWorks studio data center, combined with NetApp cloud services through Microsoft Azure, enabled the teams to iterate on simulations for surfaces such as water, set details and other special effects. That kind of functionality helped the team wrap production on time and with a remote workforce.
No data center is an island
DreamWorks Animation takes around four years to complete a feature film. Technology teams are involved from the early drafts and storyboards of the script, providing time for those teams to come to grips with what new tools or capabilities will be needed, according to Miller.
"We do a forecast of how much compute, how much rendering is going to be necessary for this film, given its complexity, the number of characters, effects and simulation, and then we work that into a budget," he said. "Things like storage, rendering and the network are all operated as a shared service. Each show gets a slice of that that changes over time as their schedule changes."
Scott "Skottie" Miller
Storage, however, forms the basis for any production's technical resource budget.
"File services I consider our factory floor, [with about] four or five hundred people continuously reading and writing files," Miller said.
NetApp's hardware and software manages the entire lifecycle of a film's data, according to Miller, ranging from preproduction to final master copy backups.
DreamWorks has been a NetApp customer for decades but has moved more workloads to the cloud due to remote work mandates during the COVID-19 pandemic. Retaining flexibility between on-premises hardware and the cloud was important for DreamWorks, Miller said, leading the team to stick with NetApp over private cloud services such as HPE GreenLake.
NetApp all-flash FAS arrays in the DreamWorks data center form the storage backbone during production, delivering what Miller called predictable and reliable performance for access and writes.
"It does enable some different workflows that can rely more heavily on storage," he said. "Now we can pre-compute a data set [for animators] and make that available instead of having to compute on it each time."
That high availability of storage also enables the artist teams to quickly pull in effects or set designs, which would normally take time to incorporate into scenes, such as the simulation of water on characters or the physical properties of the ocean.
"The water we're using in our film is not physical water, but it's directable water, it's artistic," Miller said. "The bulk of the simulation is available the whole time."
The speed of flash, alongside additional resources in the cloud, has enabled technology teams to iterate on new capabilities as needed for a project.
"It's a very iterative, give-and-take process," he said. "Much like the creating of the film, the creating of the tools to make the film works the same way."
Wave of automation
DreamWorks Animation was able to efficiently transition to the cloud due to the availability of NetApp storage software in the cloud, according to Miller.
Due to prior contracts and corporate agreements, DreamWorks uses Microsoft Azure for many of its cloud needs. Azure has enabled DreamWorks to hire remote employees, but distance creates new challenges around speed and security.
Azure NetApp Files, NetApp's Azure native file storage platform, provides greater and more consistent performance over Azure Files, the hyperscaler's own file storage service, Miller said.
"[Cloud is] not necessarily as fast for your workflows -- you do have to make some concessions to how and where and when you work," he said. "I get good, predictable behavior with on-premises OnTap, I get good predictable behaviors with Azure NetApp Files."
Using the cloud also eliminated physical tapes for backups, another COVID-19 pandemic challenge as teams weren't allowed on site to manage them. Miller said the SnapMirror feature of OnTap for cloud replication alongside object storage with NetApp StorageGrid enables recovery.
DreamWorks' shift to the cloud started around 2015, when technology teams adopted a microservices model for applications, according to Miller.
"We took a microservices approach and a DevOps deployment approach on premises as a way to get used to what it would look like to deploy in a cloud services provider," he said. "So when it came time to put workloads in Azure, it was less of a lift and shift, and more of an extension."
As DreamWorks integrates more of the cloud into its workflows, Miller said he's looking forward to using the technology of Instaclustr, a database-as-a-service company acquired by NetApp last year.
"Everything that describes asset management and data management is captured in various databases," Miller said. "The ability to deploy, operate, update and scale those databases on demand with automation is super valuable, because it means my platform and operations team can work on hard problems."
NetApp's sustainability dashboard and the anomaly detection capabilities through it cloud services are also updates Miller said he's looking forward to in order to help DreamWorks meet its own green goals and protect intellectual property.
Tim McCarthy is a journalist from the Merrimack Valley of Massachusetts. He covers cloud and data storage news.
