top of page
  • Writer's pictureValentina Zanetti

How to Measure for Success Using Agile Metrics

What to measure and what not to measure in an agile environment to make sure you're on the right track.


Agile metrics

In previous posts, we've discussed team formation and team storming. And as you exit the storming phase of your team's development, you will need to start monitoring their progress and the success of your digital products more closely. There are agile metrics you can do to gain insights into progress and the quality of your team's work, without being invasive and micromanaging your team.


But just to add a reminder, agile metrics focus on quality and flow, instead of quantity. So whichever way you choose to monitor your team's progress, make sure that this is the information you are measuring for.


There are absolutely metrics you'll want to avoid using. We'll start with those.


The Not So Agile Metrics You Should Avoid Using


In the agile sphere, metrics that don't contribute to identifying quality or flow are referred to as vanity metrics. A vanity metric is any metric that is not focused on results and delivery, i.e. the team's effectiveness and quality of results, but rather on pure efficiency and busy-ness.


These are just some of them:


🔻 Lines of Code

Not only does this metric say absolutely nothing about whether the team members yielded any actual value and whether they delivered an increment, it says absolutely nothing about the quality of the code, either. And, a large code-base goes against one of the principal values of agility - to strive for simplicity.


🔻 Utilization

Did you know that there are many studies showing that utilization of over 80% in knowledge work leads to a serious decline in throughput, actually stalling progress and stifling innovation. This is also an easy metric to fake - the team members can simply stall their tasks and finish them later to make it look like they've been working. This is absolutely a metric you can and should skip.


🔻 Number of Issues Completed

In an agile environment, this is not a useful metric, since not all issues are the same size, complexity or relevance, and your team might deliver 10x more value in a single story than in 20.


🔻 Velocity & Estimate Accuracy

Estimates are needed and valuable, but only to the development team, to be better at predicting what they're able to commit to during sprint planning. It's useless to compare it between teams or use it as any kind of individual marker of performance for a single team member. Rather, measure how often your team meets their commitments and reaches Sprint Goals.


These are just some of the metrics you should avoid because they only generate overhead without providing any actual value, as well as potentially deteriorating the agile values you've worked hard to build up in your team.


Remember, agile is about honesty, openness and transparency, knowing the real state of the matter and acting accordingly. If you force vanity metrics on your team, you will obscure these insights. Always focus on using metrics that measure quality, rather than quantity.


Agile Metrics for Team Success


But if not by quantity, how to measure how your team is performing in an agile environment?

These are just some of metrics to use when working with teams and what they mean:


🔶 Predictability

Deadlines and budgets are a reality in agile environments too, and they need to be addressed somehow. The best way to do this is to measure your team's predictability. It's as simple as looking at how frequently your team meets their commitment, and how often do they meet the Sprint Goal. When commitments and Sprint Goals aren't being met frequently, you can look for reasons why this is happening and address any obstacles and dysfunctions. In a business context, for budgeting and planning, the only metric you really need is whether or not your team can be depended on to meet the objectives they've committed to.


🔶 Flow

The first AND third principles of the Agile manifesto are about frequent continuous delivery, i.e. flow. So obviously, it's important that your team has a good flow and that bottlenecks are identified and dealt with.

Some metrics you can use to see if your team's flow is improving are:

🔸 Lead time minus cycle time, to identify where the bottlenecks are

🔸 Work in Progress, to identify whether the team is over-committing or multitasking and switching context too much, affecting their flow

🔸 DORA metrics used for DevOps, such as deployment frequency, to see how often you're actually delivering increments

🔸 Time to market, to see how often your team delivers value that is beneficial for end users

🔸 Time to recovery, to see how nimble your team is when issues arise and how quickly they can tackle them, and to measure to improve this


🔶 Quality

Working software is the primary measure of progress, so says the Agile Manifesto. And delivering working software is the primary goal of any team, not just agile teams. So how to measure for quality?

🔸 Code review and testing (manual and automated) results, to see how many times sub-optimal code is delivered

🔸 Sprint review feedback, to see how well the efforts your team has produced align with overall product goals

🔸 Defect density, to see how many bugs go into production

🔸 Performance, i.e. how well your team's deliverable is doing in the real world


🔶 Communication

Is your team growing as a team? Is their collaboration improving? Some ways that can help track that:

🔸 What level of conflict is your team on?

🔸 Are all communication lines open and is everyone's voice being heard?

🔸 How often is information not shared with the whole team?

🔸 Team satisfaction surveys

🔸 Sprint retrospective findings


Agile Metrics for Product Success


Measuring for deliverable and product success is easy, right? If the customer is satisfied, it was a success!


But in order to build really amazing products successfully, there's a bit more to consider.


Some of things that sometimes get overlooked, but should be baked into the definition of quality for all digital products are:

🔸 Stability and performance. While the feature itself might be an awesome idea that's useful to the users, if it's not quick, stable and easy to use, it will not achieve its true potential.

🔸 Impact. Is this feature inspiring the desired behavior from users, i.e. are the users using it and behaving as desired? Is it contributing to your core User Journey or deterring from it?

🔸 Sustainability and scalability. So your feature's a hit and you want to build on it. But the code is overly-complex, messy and needs refactoring, and a lot more investment and work to sustainably grow it, leaving you lagging behind the potential benefits you might have reaped with quicker expansion.

🔸 Business performance, or how is this specific feature and it's success benefiting the business? What is the gain here, whether it be financial, marketing, awareness, branding... How does this feature align with strategic goals? Does it help your business grow, or is it just fluff?

🔸 Cost/Budget variance. Is this feature worth the time, effort and financial resources invested in developing it and placing it on the market?


How and When to Apply Agile Metrics?


When implementing metrics to your teams and products, always keep in mind that metrics are indicators, not objectives! So, when applying metrics to any aspect of your business, take the following into consideration:


🔶 Never set a metric as an objective to achieve! It's counter-effective, and it will cause you and your team to focus away from the goal, and onto realizing that metric. That is fertile ground for cheating and cutting corners.


🔶 When you measure for anything, especially when it comes to team performance, always check the mean values first. And measure progress and improvement against that mean value, again, always keeping the actual goals in mind.


🔶 When applying metrics to a newly formed team, allow for a couple of months to pass for the team to stabilize before you start using metrics as any type of indicator of dysfunction or performance. Newly formed teams on new projects are always unpredictable until they get out of the storming phase, allow them patience to get into their groove.


Metrics can be a useful tool to identify blind spots, to plan and to report to stakeholders, but they're by no means an objective in their own standing. Focus on achieving goals, not metrics!


Conclusions


Agile metrics are used to measure for flow an quality, so remember to put emphasis on that when you measure. Create an improvement loop for your metrics as well, and discuss with your team is the measurement you're using actually showing you the data you want to see, and if not, check for better ways to achieve that insight. But don't stay overly focused on your metrics. Use them as indicators or good performance or of dysfunction that you can later address with your team, but never as an end-all tool. They're meant to prompt discussion, not end it.


For more insights, feel free to reach out to us at coven@agile-alchemists.com and don't forget to subscribe to our blog for more goodies directly to your inbox!

89 views0 comments

Recent Posts

See All

Comments


bottom of page