BeyeBLOGS | BeyeBLOGS Home | Get Your Own Blog

« January 2010 | Main | March 2010 »

February 27, 2010

Measure Agile Projects with KPIs

In this blog, we've been primarily exploring complex project performance methods and using BI to do so. This covers areas such as earned value, resource utilization and portfolio management. Those areas of PPM tend to get minimizalized in Agile projects.

Now let's start a high-level look at measuring KPIs and performance of software development teams that practice Agile development methodologies.

This blog is BI PPM and project managers play an important role on agile software development project as well. It is a slightly different role and there is a big difference in terms of requirements analysis & validation, up-front planning and progressive schedule elaboration.

So, naturally, KPIs that you will use in your scorecards to measure Agile teams will be different. Here I will focus on looking at team & project performance based on their primary work increments called iterations. These are also not meant to get into the weeds of classic software engineering KPIs measured by a development manager or Scrum Master such as bugs or defects and we won't consider an Agile project manager as someone to track a burndown log.

1. Velocity: Based on estimations, this will tell you how much the team will be able to deliver per iteration (typically 30 days). After the first couple of Sprints, you will be able to use that history to improve your benchmarks.

2. Number of Iterations to Complete: I think of this as a sort of ETC for Agile projects, based on time instead of cost. Use the total size of the product backlog and put that over the velocity to determine how many iterations are remaining to complete the project.

3. Number of backlog items validated: Use this to help align the progress of the project to the strategic objectives (aka portfolio KPIs) to ensure that the team is satisfying the commited backlog items and not sneaking in too much skunkworks. This will help you to make a case should you need to do the unthinkable: recommend a Sprint cancelation.

Agile101 makes some good points that measuring remaining scope and time to complete (velocity) misses out on the areas that EVM covers such as impact to costs and this also focuses solely on the team's ability to deliver against the clock, not quality.

Here are 2 very good sources for more reading on Agile project management and measuring performance on Agile projects:

From VersionOne and ccpace.

Posted by Mark Kromer at 11:30 PM | Comments (2)

February 8, 2010

KXEN keeps on going ...

That's the predictive analytics & data mining company (Knowledge Extraction Engines), not the radio station!

Last I checked, they have about 40 employees and just US $5M in revenue.

But Forrester ranked them as a leader, like SAS, they leverage PhDs in Mathematics and they have a business model that I have always liked and watched from afar in that they work with software vendors and SIs to embed their technology.

Instead of the SAS large implementation model where you have to hire PhDs to use the software, KXEN allows you to put predictions and data mining in your application, hiding the implementation details.

OK, that's it. I just wanted to say something nice about the little company that sounds like a radio station.

Back to Project Portfolio Management BI next time ... Best, Mark

Posted by Mark Kromer at 3:15 PM | Comments (0)

February 7, 2010

EMV for Risk Analysis

When I meet with customers and project teams about how they are managing and assessing risk in their projects & programs, I find it quite interesting to see the different approaches from one industry to another.

For example, utility companies do a very good job of assessing and calculating risk around potential project risks and issues. Engineering & construction "owners", or those that hire contractors to build buildings, do a decent job of capital planning and assessing risks in investments and capital budgets.

But in IT project management, where I focus, risk analysis and management of risk tends to be undervalued.

Now these are, of course, generalizations. But I wanted to point this out to speak very briefly about a concept known as EMV or estimated monetary value for risk analysis. It is a good & easy way to use the risks that you've identify in your project and to use a simple formula to place a $$ value on that risk. You can then plot the probability of the impact of those risks as I'll show below.

There is a better detailing of EMV for risk analysis here.

A summary of the steps is: Assign a probability of occurrence for each risk. Then assign a monetary value for the impact of the risk if it were to occur. Finally, multiply the probability by the impact and you get the EMV. This value is positive for opportunities and negative for threats.

You can graph this as a risk chart, a tornado graph or as a distribution of probable values as I've linked to below from Oracle's Risk Management and Hyperion's Crystal Ball. The advantages of using BI tools like these for EMV is that you can get decent analysis through Monte Carlo simulations against your project schedule or your investment schedules based on the number of sample iterations you set. I also have a sample from RiskyProject below of a risk matrix where you can plot your values in 4 quadrant from least to most likely & impactful.

Posted by Mark Kromer at 2:15 AM | Comments (4)