DevOps pioneer Andrew Clay Shafer, recently hired by Pivotal, says the PaaS provider has some work to do to get its various business units on the same page. He also discusses the hype and reality of DevOps, big data and containers like Docker.
Pivotal is trying to take advantage of three buzz-worthy technologies -- platform as a service (PaaS), DevOps and big data -- by offering enterprises a new way to build and deploy applications.
The EMC and VMware spinoff Pivotal Initiative uses technology based on Cloud Foundry, an open source collaboration of some of the biggest legacy IT vendors. While enterprises are showing interest, it's unclear how much traction Pivotal and Cloud Foundry have actually gained. In July, the company hired Puppet Labs Co-Founder Andrew Clay Shafer, who worked on OpenStack projects at Cloudscaling and Rackspace, to generate support around Cloud Foundry and to push the company's efforts around cloud computing, open source, DevOps and big data.
In this interview with SearchCloudComputing, Shafer discusses the cloud market and where Pivotal fits in.
You were an early proponent of DevOps, which has been getting a lot of attention.
Andrew Clay Shafer: DevOps is still this thing that nobody understands what it means, and that means everyone involved. When you go from the traditional IT, with a sysadmin and a bunch of scripts, to being able to configure machines, provision machines, monitor machines all through the API, then writing to those APIs and the infrastructure itself becomes more and more like software. [It] looks suspiciously like software development.
There's this dream that you're going to throw everything into Hadoop and all of a sudden it's going to transform your business.
Andrew Clay Shaferco-founder, Puppet Labs
Are there big areas where there still needs to be improvement?
Shafer:You can always improve. There's so many voices now, and especially now that every big IT vendor has a cloud and DevOps story. So watching that evolve, they didn't actually change any of their products, they just changed the title of the press release, so that's a little bit underwhelming. But, there [are] also all sorts of amazing stuff going on, and so it's just a function of being able to choose what your optics are and what you chose to pay attention to.
How does that play into what Pivotal has done?
Shafer: If you want to do these kinds of continuous delivery-style deployments, you're going to need to build some kind of a pipeline to do that. Essentially, there's still a lot of work to do, but Cloud Foundry is pretty close to what you'd need to build yourself to do that kind of deployment, [and] to support those kinds of processes and architectures. If you can take the lessons of the big Web and DevOps, and all the things we learned the hard way over the last 10 [to] 15 years and put them together in a way that not all the hard problems are solved -- but a significant number are solved, so you can focus on your core competencies -- it's going to be a huge differentiator.
You can no longer discount these approaches and these processes and say, "Oh, well that's for the cool start-up Web guys." You're going to be at a disadvantage if you're not able to keep up with that pace of innovation.
How do you see big data being utilized now and where does it have to go?
Shafer: When you see these big enterprises that have very entrenched processes and all sorts of momentum and inertia try to adopt some of these tools and some of these processes, they struggle.
There are a lot of people rushing into big data projects, and they're just trying to go through a checklist of oh, we've got to have big data. But they haven't taken a methodical approach to how they're going to apply that, so there's sort of this dream that you're going to throw everything into Hadoop and all of a sudden it's going to transform your business. In reality, it's hard enough to solve that problem, but you have to spend as much or more time understanding your domain, [and learning] how to ask the right questions.
One of the criticisms of Pivotal has been this idea that there are these tools that don't necessarily work together. Is that a fair critique?
Shafer: There's some truth to that. Part of that is the fact and function of how Pivotal was formed. You have these existing business units that were put together, and they're told to make this cohesive thing, and that takes time.
If you look at the overall story around faster delivery of software and being able to connect all these dots and change your behavior, then what you have in this portfolio [is] the big data tools … and then the Cloud Foundry tooling. If you can get that to be this coherent single flag, then there's something special there, and that's what we're working toward.
Pivotal runs its platform on containers.
Shafer: How all the apps are deployed in Cloud Foundry is on containers. Frankly, pretty much every platform as a service is that way … But now there's this new project, the libcontainer that everyone is getting behind, and we're going to be committing to … Google is committed to that, Docker is committed to that, and it will just make everyone's lives easier.
Containers aren't new. Why do you think Docker is getting so much attention?
Shafer: Docker solved a user experience problem. If you understood how computers worked and how containers worked and you had this system knowledge, then it was a no-brainer, and people were using containers. But what Docker has done is made that accessible to the average developer. And really, the inside of Docker -- this is what most people miss -- is actually the layering and image management. The layered image management is the game-changing part, and it just happens to be attached to the container.
There have been rumblings that Docker has the potential to hurt VMware.
Shafer: There are certainly things that give you flexibility with containers, but you still have to install things on bare metal before you can run containers, and if you're going to run containers in Amazon, you're definitely going to run those inside of VMs. I don't know exactly how things shift. I really think there probably is some redistribution, but … there's different isolation properties when we start to talk about the process isolation of containers versus the hardware virtualization. There's a ton of different characteristics. Other aspects of those technologies are different, and depending on what you're trying to accomplish and why you're trying to accomplish it, a VM might still be compelling for you.
