Scrum author talks removing dev hurdles, old-world thinking

Enterprises in pursuit of Agile, but clinging to the large phases and silos of an established software delivery approach, cannot obtain value from the methodology. The origin of the term Agile, after all, conveys dexterity and adaptability. And Agile, in practice, asks users to work in short iterations and across functional departments, such as test, development and security.

The assembly-line, sequential approach to code delivery, which once dominated the world of IT, is not compatible with Agile, said Gunther Verheyen, a Scrum author who refers to himself as a caretaker for the framework.

He has a deep view of what effective Agile project management should look like, and how Scrum works as a common Agile managerial style.

In this episode of the Test & Release podcast, Verheyen described why Agile should focus on product -- not project -- development. When teams focus too heavily on projects, he said, they limit themselves; the collaborative nature of Agile is better organized around a product-first approach. Agile and Scrum also encourage team members to take actions that are in the best interest of application quality and delivery, instead of waiting around for detailed instruction.

"[The Scrum] framework thrives on what we call empirical process control -- or, in short, empiricism," Verheyen said. "That's just a nice name to say that Scrum thrives completely under the process of regular inspections and adaptations."

The Scrum author and consultant also explained what skills an effective Scrum master should have, and how Scrum helps teams be more Agile.

Editor's note: Verheyen spoke with site editor David Carty. The transcript has been lightly edited for clarity and brevity.

Let's talk about Agile first. From your perspective, how advanced is Agile adoption, as far as you can tell?

Gunther Verheyen: In a way, David, to be honest, advanced and at the same time, quite confusing also, because over the years, you know, Agile came from the Agile Manifesto back in 2001, started to become really huge, sort of what we say, "crossed the chasm," around 2005, 2006. But with the success of Agile, and over the years, everybody wanting to become more Agile, [so] sort of the meaning of the word also gets lost a little bit. So, in that sense, a lot of organizations and honestly, I think Agile sort of took over the world, but in that taking over the world, a lot of confusion also arose by the meaning getting lost a little bit. So, it's sort of a double-sided story. Now, we can only cherish the success and the huge adoption obviously. It's where I feel my focus on Scrum is really helpful.

Because, if I try to explain Agile to people, I always try to go for the most simplest thing that might possibly work, which is one of the Agile principles too for me. And I've been thinking about this a lot. And it started boiling down for me to the point where I just explained Agile as just a synonym for adaptive. We often see fixed rigid approaches. Agile means adaptive. But then, how do you become adaptive? How do you do that? And that's where for me Scrum is really powerful. So, the Agile Manifesto [contains] four value statements, 12 principles, very beautiful, the meaning got lost, although it's really great, but if I talk to people and introduce Agile just as another word for being adaptive, that resonates a little bit.

It helps simplify it for those people, sure. And I do want to ask you about Scrum as well. But how have you seen that meaning of Agile misconstrued, or what are some of the misconceptions you've heard about Agile development?

Verheyen: Well, I don't know [if] it's a misconception, but with the success of Agile, and Agile taking over the world, and what I like to say, like … IT software development, product development was dominated by a sort of industrial approach, linear thinking, sequential, large phases as well, separation of people and skills. Agile tried to turn around that world, and, actually, we did. We succeeded in that, so, that's what I call, Agile taking over the world. Now, when Agile took over the world, you've got this strange, it's a very human thing. Success attracts people. And a lot of people just went for the label and not really, let's say being bothered by understanding the context, the meaning, the content behind the words.

That's sort of one of the downsides of [being] successful. Agile became huge. And then suddenly, everybody started using Agile as sort of the new prefix of things. You have Agile analysts and Agile architects and Agile this and Agile phrases and so on. And in that whole process of large quantities of people in organizations taking over that prefix, the actual meaning got a little bit lost. So, that's what I saw happening. And the misconception is, probably when a lot of people from ... Can I call it that? Sort of the old industrial world?

Let's say they felt in a way, the urgency, the need, even the pressure or the stress even to become more Agile, because Agile means adaptive. Now, who doesn't have to be adaptive nowadays, in our economies, smart, quick-turning businesses turn around, sort of things and so on? Nothing is like, stable anymore. That's sort of the feeling we get, right? So, everybody needs to be adaptive. Now, a lot of people started calling what they did adaptive without really understanding what it actually meant because what we try to achieve with Agile is becoming adaptive by working in short iterations, short cycles in time, joining forces, collaborating across barriers, across borders, certainly across silos and departments and function descriptors and so on. And, and that's sort of [the] underlying idea of how to become Agile, how to become adaptive, that gets strongly lost, certainly in large organizations. So, they started trying to be Agile, within, let's say, the old constraints, the old departments, the old silos. And that was certainly not really helpful. Now, people are slowly phasing that out, so they're getting to the real meaning. But it takes them so much time. A lot of delays, and it takes them years and years and years to get to, let's say, the true meaning of Agile and how to achieve that.

Right, and one way that you can do that is with Scrum, which you've worked with quite a bit over the years. Do you consider Scrum a more preferable option to some other approaches, whether it be Extreme or Lean Programming, and why would you consider Scrum a better option?

Verheyen: Fortunately, I love Scrum. I've been doing it since 2003. So, imagine that's going to be 16 years this year. And luckily, it's not sort of about me. It's about the world and practitioners and people facing complexity and complex challenges. And from every survey, it seems that Scrum is just the largest or the most adopted method; I call it the definition of Agile, in a way. Now, you just mentioned Extreme Programming. It's sort of fascinating, because -- do you know, David, by the way, how old Scrum actually is?

I don't.

Verheyen: Well, it was sort of born or it was sort of, actually, came into the world in 1995. Imagine [Scrum] is going to be 22 years [old] this year in 2019. So, that's 1995. That's amazingly old, right? Now, in 2001, Agile came to our world, as a term it was obviously already an English word, but it was given to something that we call Agile software development. Now, that was in 2001.

Right. That's when the Agile Manifesto came out, right?

Verheyen: Yes. So, 17 people gathered in Utah ski resort in the U.S. [at] Snowbird [ski resort], because they felt they were all representing, let's say different methods. And the two most known, let's say methods, by the time were actually Scrum, and what you just mentioned, Extreme Programming. And by the time even, certainly, in Europe, and I'm assuming sort of the same in U.S., Extreme Programming was even more known than Scrum was. But with the success of Agile, the manifesto people started signing the manifesto, pledging to it, believing in it and so on. Quite unexpected success, certainly, for those 17 people that had created the Agile Manifesto. Over the years, Scrum and Extreme Programming are still the most known ways to become or ways to be Agile, let's say. But Scrum sort of outgrew XP, Extreme Programming. So, Scrum became the most popular way of being Agile. And that just continues. The unfortunate effect of that, Scrum and Extreme Programming, they really blended  together a lot, because Extreme Programming, like the name says, had a sort of extreme focus on programming or development, how to do proper, good modern development. So, it was focusing a lot on practices, where Scrum was trying to give a sort of managerial, how to manage the work, so in a sort of management or an organizing framework. Now, you can't just organize stuff and not have good development practices in place.

That's what happened a little bit with Scrum. The focus on applying, using, employing good practices within the organizational framework, that gap lost a little bit. So, that's sort of unfortunate but we can still keep stressing the importance of that, that people, unfortunately, don't associate Scrum necessarily with great practices or great development. And that's because Scrum, by design, leaves that open for people. Scrum is like, you may know, a framework that thrives on what we call empirical process control or in short, empiricism. That's just a nice name to say that Scrum thrives completely under the process of regular inspections and adaptations. So, Scrum does no more than just lay down the sort of very bare, very minimal framework that people use to regularly inspect and adapt, inspect what is done so that they can … make proper adaptations. And the hope or the belief, but it seems to be a little bit naive, [is] that by helping people inspect and adapt regularly, that all the problems they were facing or the challenges but also the opportunities and the nice possibilities that they were seeing, that people would automatically start enforcing on them and just capitalizing on what Scrum was revealing for them. And that's not always happening. That's also an effect of that long history of industrial work or industrially organizing a world that people are not used to adapting that much. People are used to doing what they are told to do, and sort of instruction-wise and just stand there, looking at the past report out, log things, report stuff, but not act upon that. So, that's beautiful. So, Scrum is the most adaptive market for Agile around the world, bigger than Extreme Programming, and that's a bit unfortunate because the two could go very well together. But let's be happy because [it's] the most adaptive framework in the world, and it's huge. It means that millions of people around the world are using, applying Scrum in some way. I'd rather go into people that are doing Scrum, and help them get more out of Scrum rather than still having to convince them to go for something tangible like Scrum. And that's, for me, the most important advantage of Scrum. Scrum helps people to become more Agile, so, Agile being adaptive. Scrum has a very simple, lightweight, set of ways to actually become adaptive. And just go in and say, 'You now have to become adaptive,' where that was almost like forbidden for, let's say, a couple of decades -- it's not helpful. It's more helpful to say to people, 'Hey, being adaptive is really important nowadays, and here is at least a good starting point. Here's Scrum.' It's actionable, it's tangible. It's got some roles, some lightweight rules that really are very helpful for people.

Right. And it seems like organizations are getting that more and more as time goes on.

Verheyen: Yes, definitely.

You released the first edition of your book Scrum: A Pocket Guide about six years ago, and you just released the second edition. I'm curious [about] what you've seen, what has changed in that time. What are some of the new developments that you've included in the second edition of your book?

Verheyen: Well, some new developments. I don't know whether there are actually really new developments. There's at least one big finding at least, that Scrum is certainly outgrowing or transcending -- whatever you want to call it -- going certainly very strongly beyond software development. So, what we achieved which come a lot in, let's say, the first 15, 20 years of Scrum that we help people get over the idea that everything had to be organized in projects or limited in time, and so on. We help people focus on products again. So, we help people overcome the idea that Agile was just another way to do project management. So, it was more focused on product development. And product development itself is a complex work, it happens in very complex circumstances hence the need for empiricism, inspection [and] adaptation. Now, complexity is everywhere, not just in IT, not just in software development, not just even in product development. It's in all domains of society. So more important, people are faced with complexity which you could say is unpredictability, high degrees of uncertainty. And as Scrum is so huge, certainly in software and in product development in general, in other domains of society, people are discovering, 'Hey, we're faced with complexity. Here are the product development guys. They're using something that's called Scrum.'

So, Scrum is certainly going beyond, let's say, the borders of the barriers of product development into lots of other domains. So, I wanted my second edition to be a little bit more generic in the description of the rules and the purpose of the rules of Scrum because that's what I essentially wanted to do with my little pocket guide, highlight, not just repeat the rules of Scrum because they're in the Scrum guide, this short document, 19 pages only. I wanted to elaborate a lot on the purpose of the rules. Why do we have those rules in place? What problems are we trying to tackle with those rules? So, a lot about the why of Scrum, the purpose of the rules. And I wanted to make them a little bit, the descriptions a little bit more generic, than sort of closely tied into software or product development. That was one thing.

And obviously, also, in my own way of explaining Scrum, I keep evolving. I keep using different terms, different ways of expressing Scrum. So in that sense, I wrote the first version like, like you said, six years ago. The way I express and explain to people has not stopped changing either. So, it was just an update. A lot of small terms, small things. I have kept the overall structure in place, which means I try to give some sort of history, like what I just explained on the Agile Manifesto, Scrum [in] 1995, going into a little bit of the history of why we call it Scrum, actually. So, a bit of history, then going into Scrum itself. And what I've tried to do with my book, and that's the structure I'm keeping in place, help people distinguish sort of the core, let's say, mandatory rules of Scrum, what you have to do in order to call it Scrum.

And then, but that's just at the level of the rules, sort of what are we hoping to achieve with those rules, and then, distinguish that from how to apply those rules, let's say tactics to play the game. So, the rules of the game and the tactics to play the game and try to be a little bit forward-looking also, by looking into the future. So, terminology, different ways of explaining Scrum, a couple of things that I found really interesting in the classes I teach, the consulting I do, speaking at events, what resonates with people, highlight a little bit more. And so, in essence, it's a huge update because it's a lot of tiny updates, really fundamental, but still, it feels very, I feel happy for myself at least, very refreshing.

Right. And I think those conversations make sense, you know, obviously want to include what's relevant to the people that use Scrum. So, I think that makes a lot of sense. And kind of along the same note, you know, here in the U.S., Scrum Master positions and skills are on the rise. Can you explain the role of a Scrum Master, how it might differ from a more traditional project planner or administrator? And is there a sort of type of person or skill set that's a natural fit for a Scrum Master position?

Verheyen: The idea of Scrum Master [as a term] was like, sort of more like in fighting sports, martial arts and so on. It reflected the idea of mastery. So, Scrum Master is a person with some level of mastery in Scrum. Now, a Scrum Master is only one of the three roles in Scrum, but I like to call them more like accountabilities because roles is still, the word "role" still reflects the idea, the old idea of industrial things. So, a Scrum Master is somebody with mastery in Scrum, who is going to work with the two other accountabilities in Scrum, which is a product owner -- and that word should almost be taken literally, the owner of a product, which by the way, has some huge growth potential too in the application of Scrum. And then, the third role is development team.

So, what is the Scrum Master doing? Somebody with mastery in Scrum, helping those two roles, plus the outside world, understand Scrum better, understand the rules of Scrum better, help those roles and the outside world apply the rules of Scrum better, but from what we call a servant leader position. So, Scrum Master is somebody with authority, but no power. And that makes it so fascinating. How can you have two other roles and the world outside of those roles? How can you help them understand Scrum better without the ability to sort of instruct them what to do? Sort of command and control, these types of things. So, a person with mastery in Scrum, at the same time, somebody who knows when to take up the stance of being more like a coach or a mentor, or even a teacher, when to actively step in and almost intervene, when to sort of step back. And what I think is very important for a Scrum Master is, rather than focusing on the rules of Scrum and sort of be an instructor of the rules of Scrum, help people grow better relationships, the product owner and the development team, but also product owner, development team and a Scrum Master as a whole with the organization and towards the market, but certainly with managers, other teams, other people or parties that they are dependent on. And the Scrum Master is there to help everybody, within the team and beyond the team, understand and actually enact Scrum.

So, it sounds like a Scrum Master's work could vary quite dramatically from one organization to another, right? I mean, everybody's got their own organizational and internal challenges.

Verheyen: Yes, absolutely. So one of the things is, one very explicit, let's say demand or task we have for a Scrum Master is … to remove impediments. So, to go into that a little bit. So, a team, development team in Scrum and sort of at the next step, the combination of product on the Scrum Master and development team we often call it a Scrum team. So, the development team and the Scrum team as a whole are what we call self-organizing units. They thrive on self-organization. That means in Scrum, I already said that Agile is all about iterative incremental work. In Scrum, we call [them] iteration sprints. So, that means self-organization at least means that within a sprint, that collection of people and accountabilities, they organize their own work. Nobody outside of the team tells them how to organize their work within a sprint. Now, that's already a sort of dramatic change with regards to what you started your question with, describing what a traditional project leader is or almost like task instructor. There is no role like that in Scrum. A development team, as a collective of people, they decide for themselves who's going to do what, at what point in time. But all against the idea within a sprint, at least by the end of the sprint, possibly sooner, they want to have what we call a releasable version of product ready. Because Scrum, with its roots in product development, that's the goal. And the Scrum Master is there to help them understand the purpose of Scrum, become better at it. And, in that sense, it's funny that the Scrum Master is there to help them self-manage better. So, how can we, as a unit, a collection of people, how can we manage ourselves better? And that's sort of the idea of servant leadership, facilitation coaching, challenging questions.

So, Scrum Masters have lots of techniques to help a team within an organization become more self-managed, self-organizing -- except command and control. But a self-organizing unit might face problems that they cannot fix for themselves, that might transcend their powers, their abilities. And that's what we call impediments. And that's where the Scrum Master has an explicit demand from Scrum, to remove impediments, any blocking problem, anything that is hindering, slowing down the team, but that they cannot fix themselves. That's for the Scrum Master to make sure it gets removed. Impediments are often in the organization. Procedures, processes, bureaucracy, administrative stuff, that people have to go through, governance for no other reasons than producing piles of documents as well -- that is often hindering a team a lot. It's not contributing, it's not adding value. So, working on impediments, that's a very important thing for a Scrum Master to do.

Right. And it can pop up in so many different ways like you said.

Verheyen: Yes.

And let me ask you this, just my last question here Gunther. How does DevOps change the equation for Agile or for Scrum? And where do you see Scrum going in the next few years?

Verheyen: So, I already said that in Scrum we have those three accountabilities. Let's say that, one  accountability is what we call a development team. It's a group of people, and it's one role, one accountability. Their accountability is to make sure that by the end of every sprint, that short cycle in time of two to three, at most four weeks, that by the end of that sprint or earlier, they have what we call a releasable version of product. Now, we know that to produce a releasable version of product, you always need multiple skills within a development team. You need testing, documenting -- it's not just coding, lots of stuff. Some analyzing some stuff within the sprint, designing stuff and so on, all done concurrently. So, a development team has all of the skills and expertise needed to produce a releasable version of product, which … is going to be highly dependent on the context, the technology, the product, the market, the users, the organization. So, that's where Scrum already allows flexibility.

Now, in a lot of organizations, what happens is that they start using Scrum, let's say, from a sort of limited interpretation or understanding of the term development. So, there was still a huge separation between what is called development -- creating a releasable version of product -- and then operations or ops -- maintaining, supporting, logging or monitoring stuff and so on, keeping it up and running. Now, the biggest advantage of the DevOps movement was focusing people's minds, organizations, managers, CIO, CTOs, IT managers and so on, on the idea that, you know what, a successful product is not just successful development. You also have to be able to maintain and support it and monitor it. So, they wanted to focus a lot on Dev plus Ops should work together. They call it themselves a cultural thing, a mindset. And that's perfectly compatible with Scrum, because in a development team, you have to think what is a releasable version of product. And that's hopefully not just a version of product that is, has been created and produced, but is also maintainable, supportable. So, in a development team in Scrum, that might consist of Dev and Ops and lots of other skills. So, they're very compatible. So, big advantage, the DevOps movement is focusing people's ideas or minds on the fact that it's not just development, it also, operations is also required, which is perfectly compatible with Scrum.

Interesting, because you hear sometimes about how Agile and DevOps are kind of pitted against each other and, I mean, it sounds like they're more complementary than adversarial, is what you're saying?

Verheyen: Yeah, absolutely. Yeah. I know, some people say that DevOps is sort of the next phase of Agile or the next ... I think, you know what? Honestly, I think that's a lot of messaging and maybe marketing or promoting themselves. I see DevOps as, that's a go-to, that's a perfect fit for Agile. It's just focusing on a very important aspect on how to align development and operations. And that brings up stuff like automation of stuff. So, it's really cool, but they're certainly not fighting each other and they're not contradicting each other, even. [It's] the opposite. They really enforce each other. And Scrum has, let's say, the tools, the rules to actually do DevOps and be Agile.