What is a Metric?

The difference between metrics and KPIs

In the business context¹, we often talk about Key Performance Indicators (KPI). Is there a difference between a metric and a KPI?

A metric is a numerical measure used to track and evaluate performance. It is used to measure progress, recognise trends and provide insight into different aspects of a business or project.

KPIs are a subset of metrics specifically chosen to help organisations track progress towards their goals. They are typically used to monitor performance in key areas such as sales, customer satisfaction or operational efficiency.

While all KPIs are metrics, not all metrics are KPI. Opinions vary on whether KPIs are usually more important than other metrics because they are specifically chosen to help companies achieve their goals. They are also often used to monitor progress at a high level, which makes them useful for communicating performance to stakeholders.

In summary, metrics are a broader category of measures used to track performance, while KPIs are a specific subset of these, selected to track progress towards key business goals.

Risks in using metrics

While metrics can be useful to measure and improve performance, there are also potential risks when organisations rely solely on them to make decisions. Some risks are:

• They can incentivise teams to prioritise achieving certain values, even if this is at the expense of other important factors.
• Indicators that focus solely on productivity or output can tempt teams to prioritise quantity over quality, which can lead to lower customer satisfaction or an increase in errors.
• Values can only represent part of the truth, so companies may risk making decisions based on incomplete or misleading data if they do not consider the broader context.
• Metrics can create unintended incentives or behaviours that are detrimental to overall performance, for example by tempting employees to manipulate the system or inflate numbers. For example, if a Product Owner in Scrum demands that more story points be realised within a sprint, this often leads to story point inflation and the means of planning defeats its purpose.

To mitigate these risks, organisations should not use individual metrics in isolation from other information and should ensure that they are measuring the right things in the right way. They should also foster a culture of continuous improvement and learning, and be prepared to adapt surveys and approaches as needed.

Agile metrics

There are a variety of metrics that are used in the context of agile projects or developments. Below is a selection:

• Backlog Health is a measure of the overall health and efficiency of the team’s backlog, including factors such as backlog size, obsolete items and progress towards completion.
• Cycle Time determines the time it takes a team to complete a task from start to finish.
• Escaped Defects designates the number of defects that make it into the final product despite the team’s efforts to avoid them.
• Defect density denotes the number of defects per unit of work.
• The percentage of time a team actually works to create value rather than waiting for resources or approvals is called flow efficiency.
• Customer satisfaction is a measure of how satisfied customers are with the team’s work.
• The time it takes from a customer requesting a task until the task is delivered is called Lead Time.
• Mean Time To Repair (MTTR) is the average time it takes to repair a defect.
• Percentage of Rework determines the parts of the work that are reworked due to defects or other problems.
• Team morale is a general measure of team commitment and motivation, determined through surveys or other forms of feedback.
• The time it takes a team to bring a product or feature to market is called Time to Market.
• Time to Value is the time it takes a team to deliver tangible value to the customer.
• Velocity is the amount of work a Scrum team can complete within a single sprint, measured in story points or other units of work.
• Work in Progress (WIP) denotes the amount of work currently in progress in the system. This metric can help teams recognise bottlenecks and reduce multitasking, especially when combined with the WIP limit as the maximum number of tasks a team can work on at a given point.

Certainly this list can be added to. The Scrum Guide, which defines the framework with its rules of the game, does not specify an agile metric, so organisations can decide which approaches are most useful for their work.

Software metrics

There are also numerous metrics in software development. Below you will find a selection:

• Code quality measures the degree of efficiency, reliability and maintainability of the code.
• Response Time is the time it takes for a system to respond to user input.
• Code Coverage measures the percentage of code that runs through automated tests.²
• Error Count is the number of errors found in a software application during development or testing.
• Mean Time to Recovery measures how long it takes to recover from a system error or system failure.
• Code Complexity measures the degree of difficulty in understanding and changing code. This can be measured with metrics such as cyclomatic complexity or the number of conditional statements.
• Code Cuplication measures the amount of code duplicates copied and pasted within a code base. Duplicated code can lead to inconsistencies and make it difficult to maintain the code.
• Code Smells are indicators of potential problems in the code that may require refactoring. Examples of code smells include long methods or classes, inconsistent naming conventions, or excessive coupling between components.
• Test Coverage measures the percentage of code that is covered by automated tests. Low test coverage can make it difficult to identify and fix bugs, increasing the likelihood of technical debt.
• Lines of Code (LOC) is a metric that measures the number of lines of code in a software programme or system.³

The combination of individual values is also found in software development. For example, the maintainability index is a composite metric that takes into account factors such as code complexity, code duplication and code size to provide an overall measure of how easy or difficult it will be to maintain the code over time. A higher maintainability index indicates that the code is easier to maintain and less likely to accumulate technical debt.

Metrics in project management

Numerous metrics can also be found in project management. Here is a selection:

• Schedule Variance is the difference between the planned schedule and the actual progress, usually measured in time.
• Cost Variance is the difference between the planned budget and the actual expenditure.
• Planned Value (PV) is the total value of the work to be completed in a given period.
• Earned Value (EV) is the total value of work actually completed in a given period.
• Cost Performance Index (CPI) is a measure of cost efficiency, calculated as the ratio of earned value to actual costs.
• Schedule Performance Index (SPI) is a measure of schedule efficiency, calculated as the ratio between Earned Value and Planned Value.
• Return on Investment (ROI) is a measure of the return a project generates in relation to the investment made.
• Net Present Value (NPV) is a measure of the present value of all future cash inflows and outflows associated with a project.
• And Change Request Rate is a measure of how often changes are requested during a project that may affect the project scope and schedule.

In project management practice, it can also be useful to combine different metrics. For example, combining resource utilisation and resource availability can help identify areas where resources are over- or under-utilised and consequently optimise resource allocation.

Challenges in the use of metrics

Organisations face some challenges when using metrics:

• The survey should be as objective as possible, i.e. subjective influences of the measurer should be avoided.
• The determination of a metric should be reliable in the sense that a repetition of the determination leads to identical results.
• Without standardisation, e.g. in the form of a measurement result scale or a comparison scale, it is difficult to evaluate surveys.
• The effort required to define and measure values should be as low as possible. Furthermore, the continuous need to determine the values should be guaranteed, because nothing is more expensive than producing effort without a need for the result.
• Validity should be considered in two respects: Are the collected data valid in themselves and can they be combined with other parameters if necessary? It can often be observed in business practice that a conclusion from one metric to another does not really work.
• Companies should consider the meaning behind a chosen metric. Velocity in agile development, for example, is a measure for planning sprints, but it neither supports the comparison of teams nor is it an instrument for motivation or performance improvement.

Last but not least, organisations are often faced with the challenge of determining the metrics that are really useful in a specific context from the multitude of possibilities.

Lean metrics as an alternative approach in companies

An alternative approach to dealing with metrics in companies could be to focus on a few important metrics instead of trying to track and optimise all possible values. The Lean Metrics approach is based on the following principles:

• Focus on the outcome, not the output: Rather than simply measuring activities or outputs, Lean Metrics focus on measuring the impact of those activities on desired outcomes.
• Measuring what matters: Rather than tracking all possible numbers, Lean Metrics identify a few key aspects that are most closely linked to business goals.
• Visualise and share data: Lean metrics make data visible and accessible to all team members so that everyone can see progress and identify areas for improvement.
• Continuous improvement: Lean metrics are used to drive continuous improvement activities rather than simply monitoring performance.

By adopting a lean approach, companies can focus their efforts on key aspects and drive meaningful improvements without the risk of being overwhelmed by the amount of data collected.

Impulse to discuss:

In which cycles should organisations ask themselves whether the metrics they use still serve their purpose?

Notes:

[1] It is not only in the corporate context that metrics play an important role. The term is also actively used in literature, music, chemistry, physics and mathematics.
[2] Code coverage can be determined using various methods: Statement Coverage, Branch Coverage, Path Coverage and Condition Coverage.
[3] Lines of Code (LOC) can give some indication of the size of a code base, but is not necessarily a good measure of software complexity, quality or productivity. Historically, LOC has been used as a metric for software development for a variety of reasons:

• As software programmes grew in size and complexity, there was a need for a way to measure the size of codebases in a consistent and standardised way.
• LOC can be used as a rough estimate of development effort and project scope, especially in the early stages of a project when more detailed metrics are not yet available.
• LOC can be used as a benchmark for productivity comparisons between different developers or teams working on similar projects.

However, the use of LOC as a metric for software development is subject to some limitations. For example, different programming languages and styles can yield very different LOC values for the same functionality. In addition, LOC does not take into account the quality, maintainability or efficiency of the code and may even tempt developers to write more code than necessary to achieve a goal, resulting in bloated code that is harder to maintain.

Here you can find a German-language video about metrics for higher software quality.

Here you will find additional information from our t2informatik Blog:

What does t2informatik actually do? One click and you'll know.