How to create a testing scorecard

Sure -- every good IT team uses scorecards to communicate project status. Scorecards aren't magic; they're simply a concise, data-driven way of reporting status. A good scorecard, unfortunately, isn't globally applicable -- what works for one team may not be needed for another.

To create your scorecard, think about what your customer is looking for. See what it was about your requirements score card that pleased them. My bet is that the score card showed your actual progress against estimated work in the project. It probably included items like blocking issues, call-outs of critical or high-visibility work items, and some sort of burn rate analysis. Talk with your customer and find out what data they found really useful and then map that information to test status.

Your scorecard will be different in each phase of your project. Early on, your scorecard will focus on your test planning. It might include the number of test specs planned, actual delivered, and a (rough) estimate on percentage complete. For instance, in my work at Circuit City, at any given time we probably had five or six test specs being written concurrently. For a given project phase, then, we would report Expected: 6, Delivered: 2, Est. complete: 65%. The estimated complete percentage is basically a guesstimate -- of the 6 test specs due, how far are we, considering the number delivered and the progress made on the outstanding specs. Your scorecard will also track your test case development -- estimated test cases (total), actual test cases written. If you have a long planning phase, you may want to track estimated total, estimated percentage for the current week, and actual.

During the execution phase, your scorecard should shift focus. You'll report out on test case totals, cases executed, pass/fail rates, etc. As performance testing kicks off, you may want to add a separate scorecard for that activity. This is where the customer is going to be focusing in on your performance so you will want to be crystal clear. Resist the urge to try to hide poor progress, too. If your team's estimated on the time to complete each test case was low, call it out in the risks/issues portion of your score card. If the incoming bug rate is high, or if most of the bugs are high-severity blocking issues, call it out. You want to deal in facts -- all of the facts.

In the past, scorecards I've produced for testing have included the following information:

Plan Phase

• Number of test specs expected
• Number of test specs completed
• Percentage of test planning completed (est)
• Number of design docs outstanding (indicates where a test spec is blocked due to lacking documentation)
• Number of open positions
• Number of positions filled
• Number of testers hired in the week
• Risks
• Issues (things that block progress on the above information)

Create Phase

• Total number of test cases (estimated)
• Total number of test cases completed
• Total number of test cases ready for review by SMEs, dev, etc.
• Number of open positions
• Number of testers hired in the week
• Risks
• Issues (things that block progress on test case design)

Note that if, at the end of the create phase, there is a delta between the estimated and actual test cases, you'll want to call that out on the final report cards for the phase.

Execute Phase

• Total number of test cases
• Test cases executed
• Goal of test cases to be executed at this date
• Percentage complete
• Goal of percentage complete
• Defects reported to-date
• Risks
• Issues

Execute Phase: Defect Score card

Project Wrap-up
At the end of the project, you'll want to create a summary scorecard. This card should report out on cost -- estimated versus actual. Show the customer what their money bought them. One card might show quality metrics (bugs found, bugs fixed, etc.). Pick up summary highlights from the final scorecard of each phase and merge them.

Stephen Seay has a fantastic article on scorecards.

Templates and explanations about scorecards abound; a simple Google search on "project scorecard" will return a number of great results, including pre-formatted templates and guidelines. When you're done with your scorecard, send it our way so we can see the great work you did!