Most of us run experiments as part of the validated learning principle. Even more (corporate) startups use the pirate metrics to measure customer engagement.
But how can we effectively measure our innovation activities? How can we compare one corporate startup to another? Is that even possible?
While working with hundreds of startups and corporate innovation labs in the last few years, we started to develop our own innovation accounting framework. Because we started to see patterns and felt the need to make the progress of startups measurable and transparent.
We use this framework, for example, as the basis for GroundControl, our software platform to help corporate startups to innovate in a structured and measurable way.
In this article, we dive into how you could start with innovation accounting in practice today. A good start to learn more about innovation accounting is our blogpost on the what, where, and why of innovation accounting.
Types of Innovation KPIs
What we learned is that there are three types of key performance indicators (or KPIs) each company should be tracking to measure innovation performance:
Reporting KPIs are connected to Innovation Practice. These focus on startup teams, the ideas they are generating, the experiments they are running and the progress they are making from a great idea to scale. The questions these KPIs answer are: How fast and how well are my startups innovating?
Governance KPIs are connected to Innovation Management. The focus here is on helping the company make informed investment decisions based on evidence and innovation stages as part of Portfolio Management. This answers: How well is my Innovation Lab, program, or Accelerator doing?
Global KPIs are connected to Innovation Strategy. The focus here is on helping the company examine the overall performance of their investments in innovation in the context of the larger business. What is the impact of our innovation on the overall strategy?
The Reporting KPIs (or startup team KPIs) are the innovation metrics that fuel the whole innovation accounting framework. Without teams, there is no accounting possible. Just like there is no financial accounting possible, without selling anything.
So to start with innovation accounting, it is important to have teams on the ground, running experiments, and interacting with customers. We’ve seen too many corporates that started to develop an innovation accounting framework first, without any teams and it just doesn’t work. You are building a theoretical framework with no way of testing your assumptions. (Sounds a lot like waterfall product development, right?).
3 Reporting KPIs to start with
How to start with innovation accounting? We start with measuring the following three innovation KPIs from corporate startups:
1. Experiment Velocity
The experiment velocity tells you how fast a team is able to run experiments. We measure the number of experiments a team is running per quarter and divide that by the number of experiments the team could have performed. Usually, this is one experiment per 2 weeks, so 6 per quarter. This rate tells us the relative and comparable execution speed of the team.
If the team has run 4 experiments in the last quarter, their experiment velocity is 4 out of 6 or 67%.
The experiment velocity only tells you how fast a team is innovating, not how well they are doing it. It is, therefore, an activity metric. The impact metric counterpart is the learning ratio.
2. Learning Ratio
The learning ratio tells you how fast the team is learning.
We measure the number of successful experiments and divide that by the number of executed experiments.
If the team has run 4 experiments in the last quarter, but only 1 was successful, the learning ratio is 1/4 or 25%.
The big difference with the experiment velocity is that we look at the number of successful experiments. An experiment is successful when it has clear learnings within the learning goal that we set upfront. The outcome of the experiment can be both a validation or an invalidation of a hypothesis. An experiment is only a failure if the team hasn’t learned anything. The team then basically wasted their time by either doing the experiment wrong or by doing the wrong experiment.
The Learning Ratio is an impact metric since it tells you how well the team is innovating.
3. Team Happiness
The final innovation KPI we keep track of is Team Happiness. We track the Team Happiness at the end of each innovation sprint, as part of the Retrospective.
The Team Happiness score is a “culture metric” and is a subjective measurement: the teams themselves decide how happy they are with their work of the past two weeks. It is a clear leading indicator of how well the team is performing. It shows you rather quickly when things go downhill, much faster than the Experiment Velocity or Learning Ratio. Usually, first, the team Happiness drops, after which the experiment velocity and learning ratio start to drop as well.
After a month or two, you should already see which teams are working hard and which are learning fast. It will also become clear why some teams make more progress than others. When you have these team metrics in place and make them actionable, you can continue by looking at how long each of your startup teams takes to progress through your stage gates.