How much is your R&D worth? The answer lies in what you do not count

A seemingly simple question

Imagine a management meeting. At some point, the chief executive turns to the team: “In concrete terms, how much is our R&D worth?”

The finance director has the figures. The head of research knows the annual budget, the number of patents the company holds, the publications its teams have produced. Enough for a solid presentation, with curves and year-on-year comparisons. And yet something is missing. These data tell us how much the company invests, what it protects and what it publishes. They do not necessarily tell us what it is really capable of doing. Still less whether it will be able to keep doing it in five years' time.

My training in molecular biology no doubt shapes the way I look at this question. When you study a cell, its mass or its size is not enough to understand how it works. What matters, too, are the mechanisms, the interactions and what flows between its different components. Why would we look at a research organisation any differently?

What the metrics measure… and what they leave out

Take the budget first. It measures the effort a company is prepared to make. This is essential information for following its research policy. But a large budget does not, on its own, guarantee large results. Two teams with the same resources can follow very different paths: one opens up a new avenue, the other gradually improves an existing process. The amount invested gives no way of weighing their respective contributions.

Then come patents. They tell us about the inventions the company seeks to protect. But filing a patent and actually managing to exploit an invention are two different things. Surveys of inventors show that a substantial share of patents never leads to commercialisation. Some remain unused, neither sold nor licensed. The portfolio can therefore grow without every patent in it finding an application.

As for publications, they reflect scientific activity and its recognition by peers. That does not make them an absolute guarantee of reproducibility. In a survey published by Nature in 2016, more than 70% of the researchers questioned said they had already failed to reproduce an experiment carried out by others, and more than half had failed to reproduce one of their own. There is also a difference between producing a great deal of knowledge and producing knowledge that profoundly changes a field. A study covering 45 million papers and 3.9 million patents, published over several decades, observed that they have become less disruptive over time. The number of publications is rising, but this growth does not mean that each one brings a breakthrough.

None of these findings makes traditional metrics useless. They simply show that we should be careful not to ask more of them than they can really tell us.

In summary: what each metric measures

Looking at what keeps R&D alive

It was from this line of thought that the ICRI, the Invisible R&D Capital Index, was born. Its principle is to look at dimensions that contribute to an organisation's research capacity but that the usual dashboards struggle to capture. It examines five of them: critical skills, that is, the know-how on which the continuation of certain projects directly depends; the retention of the people who hold them; the reproducibility of results and procedures; the partnerships that still give rise to genuine exchanges; and finally dormant assets, whether data, prototypes or intellectual property rights that could be valorised further.

These dimensions cannot be reduced to a budget line alone. Yet they influence the continuity of the work, the ability to pass on knowledge and the prospects for future valorisation.

A company can look solid and still be vulnerable

Let us take a fictitious but entirely plausible example: an industrial SME with eighty employees. Its research spending is stable, its patent portfolio is growing and its monitoring indicators worry no one. Looking more closely at how it works, however, we discover that its most important process depends on one experienced technician. She has mastered it perfectly, but some of what she does by hand has never been formalised. Two university partnerships appear in the company's documents, although no substantial exchange has taken place for eighteen months. And a database built from several years of trials sits unused on a server. None of this shows up immediately in the number of patents or the annual budget. Yet the departure of this technician could put an essential activity at risk.

The opposite situation also occurs. Some companies file few patents because they prefer to keep their manufacturing secrets. Others rarely publish, as their teams are mostly busy with industrial development. That does not mean they have little knowledge. Their skills may be well distributed, their procedures documented and their collaborations particularly active. Besides, the value of a piece of research is not always revealed at the moment it is produced. Researchers have studied the trajectories of millions of publications and identified works that were rarely cited for a long time before attracting considerable interest, sometimes years later. They are called the “sleeping beauties” of science. One more reason not to confuse immediate visibility with long-term potential.

Risk becomes concrete once it has a name

Sometimes a very simple table is enough to bring a vulnerability to light. In one column, the essential know-how. In the other, the people able to put it to use. If a strategic skill is mastered by only one member of the team, the nature of the question changes. The R&D manager probably already knew that this person was valuable. What they had not necessarily measured was the extent to which the continuity of a project, or even of a whole activity, rested on that person. What has just come to light is not the skill but the organisation's dependence on that skill.

Five questions to get started

There is no need to launch a full diagnostic straight away to start looking at your R&D differently. A few questions are already enough to identify points that deserve particular attention.

1. If someone left the company tomorrow, which experiments or processes could no longer be carried out?
2. Which skills are we at risk of losing, and how are we organising their transfer?
3. Could a new recruit reproduce an important procedure using only the documentation available?
4. Of the partnerships we have signed, which still translate into concrete exchanges or work?
5. What data, inventions or prototypes do we own without exploiting them, and could they interest other organisations?

The answers will not immediately put a monetary value on R&D. They will, however, give a clearer picture of where its strengths and weak points lie.

And where does the research tax credit fit in?

When these subjects come up, one objection is often raised: “We already have the research tax credit. And we file patents.” It is precisely because these schemes serve other purposes that they are not enough to address the question. The research tax credit (crédit d'impôt recherche, or CIR, in France) is a tax mechanism linked to certain eligible R&D expenditure. A patent, for its part, offers legal protection for an invention under certain conditions. Neither of them tells us directly about the transfer of know-how, dependence on one person or the real activity of a partnership. You can protect a formulation with a patent without having properly documented the hand movements needed to stabilise it. You can fund a series of trials without organising how the data produced will be kept and reused. These are different issues, which call for complementary tools.

Three actions to put in place on Monday

The first is to list the know-how essential to current projects and to identify the people who have mastered it. Every skill that rests on a single person deserves a closer look.

The second is a transfer test. Give an important procedure to someone who did not design it, with only the available documents. The difficulties they run into will show what remains implicit and what should be better recorded.

Finally, go back over the inventory of your patents, datasets and prototypes. Which are in use? Which could be valorised? Which have been forgotten? This last group may contain opportunities, even if not all of them will turn out to be exploitable.

Does this mean abandoning the classic metrics?

Certainly not. Budget, patents and publications remain necessary to monitor an activity, make relevant comparisons and meet certain reporting obligations. In organisations where skills are well shared, methods properly documented and dependency risks already monitored, they may even be enough for some management decisions. The problem arises when these figures are used to answer a question they were not designed to address. Assessing the level of investment is not the same as assessing a team's ability to preserve, mobilise and pass on its knowledge.

A study that particularly struck me

In 2012, two researchers published a commentary in Nature on the difficulties of reproducibility in preclinical cancer research. They reported that, of 53 studies considered important and examined by their industrial team, only six could be confirmed under their evaluation conditions.

This paper struck me, no doubt because it touches on my original discipline. The studies in question had been published in scientific journals and had attracted interest. That did not, however, guarantee that their results could be independently reproduced. Publication and reproducibility are two different dimensions of scientific value.

What you do not see at first glance

We naturally tend to trust what comes in the form of figures: an amount, a number of patents filed, a publication curve. These elements are visible, comparable and easy to include in an annual report. But R&D also holds together thanks to things that are less immediately measurable: experience acquired over the years, a method clear enough to be passed on, a collaboration that helps to unblock a problem, or data for which no one has yet imagined a new use.

So the question is not only how much a company devotes to its research. It is also to understand what it has really built with that investment, what it risks losing and what it could still do with it.