Improving Begins with Measuring

Most of us have at some point finished a project or changed a process only to end up unsure whether we actually improved anything. This is why measuring progress is so critical. That said, there are also plenty of pitfalls in the process.

The only way to know for certain that we’re making actual progress is to put a plan in place and include how we can measure its value. In other words,

“That which gets measured gets managed” - Peter Drucker

Or more simply,

“If you're not assessin’ you're guessin’” - Unknown

When we measure, we uncover the current state of things as well as gain a better understanding of how improvements play out throughout our new plan or project. Often, these take the form of KPIs (key performance indicators).

Outputs VS Outcomes

The first step of effective measurement is deciding what to actually measure. At the most basic level, this comes down to outputs vs outcomes. These can range from the number of widgets produced (output) to the count of errors or flaws per a set number of items produced (quality).  

We can get a general idea of whether we’re improving by how we feel (qualified measurement), but if we want specific results then we need a more specific metric to track. In each instance, there are output and outcome metrics that help us to understand the measurement of success.

How to Think About Outputs

Output is the measurement of quantity. This could be the number of tasks completed in a project or the number of projects a business completes in a year. Often these are the easiest metrics to capture. However, they are also often the most misunderstood. Outputs are a good thing to know most of the time. But, they need other metrics to be useful.

Going back to the count of items produced. If we have a project that increases product output, then having a count is going to be great. However, it does not demonstrate a complete picture of events. For example, an increase in output can be accompanied by a decrease in quality. This is where outcome or results-based metrics come into play.

The Challenge of Getting Measurements Right

Determining which metrics to measure in a process might take some experimentation.

The Project Management Institute (PMI), for example, uses what they called the triple constraint (also known as the iron triangle) for management and a basic understanding of measuring for projects. The three constraints are the scope, schedule, and budget of the project.

They put boundaries on those three to understand when a project is going to exceed or fall short. However, since these are often only output metrics, there was no consistent eye on quality, outcomes, or results.

In other words, this approach is helpful in some instances but not an ideal mechanism for establishing the right outcome. The underlying philosophy is that projects usually go off the rails due to a lack of constraints and discipline allowing for little flexibility or adaptability which is required in some project environments.

Enter: Agile and Its Misinterpretation

This is where the Agile Manifesto comes in (note this is the manifesto for software development, but you can also find versions for marketing, HR, and more). Agile practitioners now had a set of values and principles to inform how projects should be managed. However, some people mistook this as a call for the complete abandonment of discipline.

Agile focuses on measuring value instead of outputs by placing a strong focus on stakeholders. In other words, it presses you to think about who you’re doing the work for and how you can best deliver something valuable to them. In practice, this translates into a strong emphasis on outcomes over outputs. However, this can be misinterpreted.

One team I worked with briefly, assumed this meant they could ignore documentation and planning entirely: to the point of not even developing a long-term goal outside of the immediate requests.

Without a clear plan, some level of constraint, and discipline, the work produced by this team was haphazard, and consistently had issues delivering a goal that was not quite in focus. In the end, the loss of one key team member resulted in months of rework and the loss of critical institutional knowledge.

Finding Balance

Too much structure or not enough? That is the question.

To find the balance, we must be prepared to experiment with structure and adaptability. A clear understanding of the constraints, as well as the points of adaptability, are crucial if we want to be successful in an endeavor.

Some questions to consider when we are ready to begin measuring outcomes.

• What am I measuring?
• What is my goal, objective, or desired outcome?
• What could go wrong? How would we know?
• Is there a way to know before it happens, during, or only after?

The answer to maintaining balance in measurement is to make sure it is done proactively. This is where the importance of the last Principle in the Agile Manifesto (Responding to change over following a plan) comes into play. We need to have a plan in place to regularly review and adjust our work as well as our measurements of that work. In addition to the need to adjust until our current plan works well. What works now, might not work forever.

In Summary

Measurement is good, necessary, and not always easy. Because it is challenging there will be people who don’t want to do it. Ultimately, it can be hard to do but that should not stop us from doing so if we want to be successful.

“Success is not achieved by accident.” – John Maxwell

Key takeaways:  

1. Be aware of various aspects of the problem you are trying to solve. A little bit of research is usually enough to get you started knowing the points of leverage within any initiative.  
2. Know how and when to use output or outcome-based metrics. Are we tracking to improve productivity or the value an effort is providing? Do we have/need an understanding of both?
3. Keeping things in balance requires a regular pattern of review and adjustment. If we have a perfect mix of metrics right now, things can change, and we must be certain we have dedicated time to fix things as they change.