The pace of business innovation in a world of change

The UK’s productivity nightmare

There is agreement across the political spectrum that the incoming government will have one overriding priority: returning the UK economy to something like a normal rate of growth. By now we’re all pretty familiar with the story that, for the last 15 years, productivity growth in the UK has been anaemic, depressing national income and living standards. The scale of the problem is always worth restating and is illustrated starkly in Figure 1 below. Among the solutions, politicians concur, is more investment in innovation. But what if the confidence we place in innovation to haul us out of the mire is misplaced? This article explores the evidence.

Figure 1 UK productivity (output per hour worked) index, 1971 to 2023

Source: Frontier Economics based on Office for National Statistics data

Note: Index set at 100 in 2019

Productivity, here measured as output per hour worked, increased by more than 2% a year on average between 1971 and 2007. Since 2008, that growth rate has slowed to less than 0.5% per year.

The ‘productivity gap’ – the difference between actual productivity and what it would have been had it continued to grow at pre-2008 rates – now stands at almost 36%. Put another way, GDP per person in the UK would now be more than £11,000 a year higher had we been able to maintain previous productivity growth rates.

If the problem is familiar, so too is one of the solutions: that the UK needs to be more innovative. Productivity measures how well we can turn inputs – people, land, infrastructure, knowledge – into outputs sold at home and overseas. Innovation is the key driver of productivity. It is innovation that creates both the novel goods and services that people want to buy and the new processes that produce them more efficiently.

The policy solution is therefore often framed as we need to invest more as a country in innovation: more R&D, more labs and research facilities, more datasets and more mechanisms to help researchers and businesses collaborate on innovative ideas that have commercial potential. The 2021 Spending Review confirmed plans to increase public R&D investment from £14.8 billion in 2021-22 to £20 billion by 2024-25, an ambition the government restated in 2024. Party manifestos ahead of the election also included pledges here, with the Conservatives aiming to increase public R&D spend to £22 billion by the end of the Parliament, and the Liberal Democrats looking to boost total R&D to 3.5% of GDP by 2034. Labour’s manifesto was notably silent on explicit R&D targets.

Less for more: the productivity of innovation investment

But there could be a problem. What if innovation itself is getting less productive? What if we need to spend ever more on innovation inputs to just to keep innovation growing at the same rate as before?

This was the hypothesis of a fairly recent paper by leading academics in the US. Using both macroeconomic data and case study examples from specific industries, they argued persuasively that “ideas are getting harder to find”.

Where the US leads others likely follow. Sure enough, duplicating the macroeconomic analysis as closely as possible, we find a similar story for the UK.

The best measure of ‘innovation inputs’ is the amount spent by business on investment in intangible assets – things like R&D, software, firm-specific training and design. We now have data from the ONS on this investment going back 25 years. And firms are indeed spending much more on intangibles. Real-terms investment increased from £135 billion in 1997 to £209 billion in 2021, a rise of 54%.

In line with the US study, we convert this spending into a gauge of the number of effective researchers by dividing intangible investment by average wages for educated workers. Unfortunately, UK wages data isn’t broken down in the same way as it is in the US. However, taking a reasonable proxy (the 75th percentile of average full-time annual earnings), we estimate that the number of effective researchers in the UK has increased by 30% since 1997.

In short, ‘real inputs’ into innovation have grown a lot in the last 25 years. Productivity has nevertheless stagnated, suggesting that, as in the US, innovation in the UK has become less productive. This is illustrated in Figure 2, which charts indexed measures of the effective number of researchers over time against a measure of multi-factor productivity that adjusts output for changes in labour and capital inputs.

Between 1997 and 2008, the two measures track upwards. Following a rupture around the financial crisis, the effective number of researchers starts to track strongly upwards again in the early 2010s, but productivity remains stubbornly flat.

Figure 2 Effective number of researchers and multi-factor productivity, UK, 1997 to 2021 (indexed to 100 in 2008)

Source: Frontier Economics calculations based on ONS data

Note: Effective number of researchers is a measure of business investment in intangible assets divided by the 75th percentile of the annual full-time earnings distribution (approach adapted from Bloom et al., 2020)

Other measures tell a similar story. For example, R&D sectors are an increasingly important part of the UK business landscape. Between 2010 and 2023, the number of R&D-focused firms in the UK grew by 70% (from 10,125 to 17,235), compared with a 24% increase in the number of businesses overall.

The data hints at a further worrying sign: even spending on intangibles appears to have been slowing down a bit in recent years, in line with recent academic research findings. We will need to wait for more data to assess if this is a new trend or a blip.

Necessity is the mother of invention

So the productivity of innovation seems to be slowing down. Perhaps, though, a note of caution is needed. We know that, when push comes to shove, innovation can happen in a rush. As spotlighted in previous Frontier analysis, during Covid-19 many businesses had to innovate quickly to survive, overhauling business models, organisational structures and IT systems to enable working from home.

Innovation, then, can occur rapidly, but that doesn’t necessarily mean it will show up as extra output. Certainly, the pandemic-induced burst of innovation hasn’t materialised so far in aggregate productivity figures. Uncertainty seems to be here to stay. Whether it’s political risk, climate risk or shifting trade patterns, firms are likely to have to continue to adapt to fast-changing circumstances. If that means businesses are increasingly running to stand still, then innovation may become even less productive than it is now.

Taking a more positive view, it might just take a long time for innovation investments to translate into measured gains in output. Economic theory suggests potentially transformational technologies such as computing and, more recently, generative Artificial Intelligence (AI) are subject to the “J-Curve” effect. The idea is that firms need to spend a lot on complementary intangible investments in order for these technologies to really transform their operations. In other words, the measured returns can actually be negative for a time before the benefits are seen. Just think of the spending on software, training and management needed for businesses to re-organise themselves effectively to take full advantage of the opportunities presented by generative AI. 

Innovation in innovation policy

What does this mean for governments keen to end the productivity malaise?

Well, if innovation is indeed becoming less productive, that doesn’t mean the first response should be to spend less public money on innovation. But it does mean that government should think as much about the transmission mechanism from innovation inputs to productivity outcomes as it does about the sums being spent. What are the market failures and barriers blocking the pathway to increased economic activity? How can the benefits from investment innovation be realised more quickly?

In seeking to answer those questions, the new crop of ministers should focus on four distinct but complementary work streams.

  • Discovery – turning research into practical ideas for business. Encouraging collaboration and the transfer of knowledge is critical. In a post-Brexit world in which barriers are rising to trade and investment, it is more important than ever not to shut off the flow of ideas. This is surely an area where AI can come into its own. We already explored a few years back how automated tools can be used in academic publishing to identify opportunities for innovation. The technology has advanced by leaps and bounds since then, enabling generative AI for example to potentially shorten discovery times in medical research.
  • Translation – adopting ideas. Coming up with a new idea is one thing; putting it into practice is another. Businesses need to enhance their capacity to incorporate innovative processes, in particular by strengthening management skills. They also need easier access to innovation finance at home and abroad. It’s notable that all the party manifestos emphasised access to finance as a key focus.
  • Testing – making it easier to trial things and quickly learn what does or doesn’t work. Ministers should promote regulatory ‘sandboxes’, which give firms the chance to test ideas without being tied down by excessive red tape. And making quasi-public goods like data and publicly-supported testing and validation infrastructure – both physical and digital – available, accessible and well-publicised will be important.
  • Diffusion and adoption – recognising that spillovers are a key source of growth. The returns to society from business R&D spending are about twice as high as the private returns to the firm doing the investing. That makes it critical to maximise efforts to spread the fruits of innovation as widely as possible. But there is no obvious Departmental home for this at the moment, with science and innovation policy housed in DSIT and small business policy housed in DBT. This makes cross-government co-operation vital.

Conclusion

The need to revive the UK’s productivity growth is an economic and political imperative. Improving public services will be close to impossible without the extra tax revenues that faster growth generates. Increased business investment in innovation is regularly assumed to be a sure-fire way to boost productivity, but we have reviewed worrying evidence that innovation is generating fewer benefits than before. The picture might improve as businesses learn how to incorporate generative AI into their operations. In the meantime, Ministers should concentrate on identifying and removing any obstacles that prevent investment innovation from feeding through into increased output.