We were the first to use propensity score matching to figure out the impact of the UK government’s innovation support on business performance. We found robust evidence of positive effects on employment.
How much grunt for your grant?
The UK government invests almost £2 billion each year in business R&D – often in the form of grants to support particular innovation investments and research projects conducted by business. But there is surprisingly little robust evidence on the value for money of this spending. Does the spending make a difference to the firms that get the grants? Do they perform better, increase their revenues, or employ more people? And if so is this as a result of the investment or something else?
BEIS, Innovate UK and the National Measurement System were interested in understanding this so they could get a better handle on the value for money of grants for innovation support, and whether or not the effects depended on the type of investment or the type of firm receiving the investment.
Out of control
We used cutting-edge econometrics and big data to answer the client’s questions. We were provided with a detailed list of all the companies receiving innovation support from Innovate UK and the NMS over five years – this was our ‘treatment group’. We matched this with government administrative data on business performance.
We then used an econometric technique called “propensity score matching” to identify, among those that had not been supported, the firms which were most observably similar to the treatment group. This was our ‘control group’. We tracked outcomes for the two groups up to five years after getting the support to see what happened and compared the trends for the two groups. At the time, nobody else had used this method to look at the impact of innovation support on business performance.
We worked closely with one of the leading academic practitioners of the approach to ensure that what we did was robust, leading-edge and intellectually sound.
Surviving, then thriving
We found that receiving support made it much more likely that firms survive, and strong evidence of positive effects on employment. We also found that turnover increased, but the results were a bit less robust.
As we were the first to use these methods in this context we uncovered some methodological challenges that have improved our and our clients’ understanding of the strengths and limits of these cutting-edge approaches to estimating the impact of policies on businesses. For example, the method was very hard to apply to large firms where there is no counterpart ‘control firm’ – meaning a range of methods will be needed to get a holistic picture of the impact of these large-scale funding programmes.