Ease conflicts with software engineering project management

Beware the feature monster

The feature monster is the familiar, restless push to add more bells and whistles to an application, popularized as a concept by Ken Pier, senior director of product quality at Socialtext.

Software product features are quantifiable. If the product owner gets 10 features instead of five, he can build a feature-comparison matrix that shows more checkboxes for the product over its competitors. Meanwhile, application qualities such as usability, scalability and reliability are more difficult to measure and market to customers. Management's desire for feature-rich products can motivate programmers to take shortcuts to cram in a few additional capabilities at the expense of software quality.

The project manager has no way to measure a small decrease in usability, which makes the trade-off for a new feature seem like an easy decision: Postpone technical work and gain additional business value first. However, it's likely the next sprint will be equally schedule-challenged, so the technical work never gets done. Worse still, the backlog of improvement items keeps growing.

With this approach to software engineering project management, a lack of focus on under-the-hood improvements eventually makes the UI unusable or services unreliable. Perhaps development efforts shift to focus on quality and the organization learns a lesson -- or perhaps not.

The estimate-free option

To combat the feature monster, some engineers choose to commit to nothing, as long as it doesn't add too much risk to the organization. Or, if they must commit to the immediate deliverable, they can use safety language, such as, "our goal is April 1," or, "if all things go well, we'll have the work done at the end of February," to push the risk back onto the project manager and senior management. However, this approach removes some transparency and accountability.

In a few software engineering environments, a low- or no-estimates approach to project management actually works. For example, a company with a SaaS that is already wildly profitable might be more interested in prioritizing new capabilities than in set releases. In another scenario, a team might work entirely on bug fixes, with management counting the bugs fixed per month rather than following a release schedule.

These examples are few and far between. At most organizations, developers and the business side must come to a consensus about schedule, risk and feature tradeoffs.

Break the work down into user stories, then use a metric -- such as stories per month -- to forecast how much work the software engineering team can do. This approach takes the commitment out of the programmers' hands, but it won't solve the problem of technical work backlogs.

Save a percentage for infrastructure

Some of my clients reserve a percentage of work in the scope of the software engineering project for infrastructure management. Typically, 10 to 20% of the stories involve tasks such as rewriting a subsystem, creating a test automation suite or a replacing a web server.

This approach has two main problems. First, when you call out technical stories in the same manner as user stories, it tends to lead to implied technical projects. A team that wants to rewrite a major system might put two stories on the board per sprint and plan the rewrite over 10 sprints. This undertaking violates the principle in Continuous delivery that code should always be ready to ship. The rewritten code goes into a separate branch, where it loses track of the other changes on the trunk. Technical projects separate from the main software engineering work can create a new set of bugs that the team didn't plan for.

Perhaps worse, exposing the under-the-hood upgrades to the business side as an isolated story gives them the option to eliminate the infrastructure work in an emergency. The business side will always prefer that the project to meet its deadline, so there will always be an emergency. And when the business folks invoke the emergency rationale, they end up cutting the technical backlog out of the scope of work. That brings us back to where we were at the beginning.

One of my clients reserves a percentage of project capacity for infrastructure management as a foundational way of working. This client made an effort to institutionalize this technical work focus within the company culture, complete with posters and slogans reinforcing the idea. However, culture can change.

Separate engineering tasks from management

In my experience, the CIO can eliminate debate and compromise in software engineering project management, particularly concerning project priorities, with a simple directive: Let engineers be engineers.

For example, look at how civil engineers calculate the load and determine the materials for a bridge. When they put their signature on the plan, they assume legally responsibility for that bridge. If that engineer refuses to sign, the bridge won't get built.

Programmers should have that same capability; they should be able to refuse to make a hack that compromises reliability, usability and scalability. If a web server needs an upgrade, rather than request permission from the business to do it as a technical story that needs funding, the engineering director can build this work into the cost of producing software. Robert C. Martin, co-author of the Agile Manifesto, called it the "Boy Scout Rule" to leave the code in a better condition than you found it. Pay the 5% or so in time required during the project to have the programmers clean up the code as they go. If programmers need stories that must be tracked and take a large percentage of time for technical work, that is indicative of a skills problem.

There are exceptions to the idea that programmers should prioritize software quality over feature development. If the company might go out of business in two weeks, and a demo tomorrow could secure more funding to prevent that, take the quick and easy way to get the demo running and leave technical fixes for later. True emergencies of this nature can occur, though these occasions are rare in real life.

Partner with the business

Typical companies work according to a roadmap set for anywhere from two to five years, though it is nothing near solid. Half of the items on a corporate roadmap never get done, the other half will be late; and half the things the group does over the next two years are not on the original roadmap. That's all ok, even expected, as mature managers and executives in high-change environments have realistic expectations that balance risk, predictability and output.

While it may not be the most tempered or considered approach, the technical staff can create a roadmap that is simply the cost of doing business -- the work that makes sure the foundation is secure. But this endeavor requires principled stances and even abject refusal at times. This is where things get messy.

Instead, the technical staff should track their own deliverables separately from the backlog that is exposed to the business, and deliver improvements along the way. In the end, this method provides the true cost of doing business, including infrastructure work, which leads to a more accurate roadmap.

It's easy to justify this roadmap. Simply find out what the business side wants to accomplish with each software engineering project -- to attract new users, increase product velocity or another goal -- and then explain how the software must change to make that possible.

While it makes some sense to keep the product decisions with the business and engineering decisions in engineering, it's simply not that easy. Technical staff often has insights that lead to fantastic products, and management can have product plans that affect infrastructure. The important thing is to keep the relationship balanced; as soon as either side takes advantage of the other, start a clock, because in a year or so, both sides will lose.

Software engineers, make a technical plan that includes support, upgrades and changes needed to support new users, then refactor code as you go. Keep that list separate from the business-feature list, but work collaboratively to weave the two together.