R&D Performance Measurement Systems – Valutrics

The significance of R&D for industrial organizations is often described as one of the most important factors affecting future profitability. Having in mind the four schools of thought: (1) financing/ funding, (2) business strategy, (3) business processes, and (4) employees, we explore its applicability to R&D.

R&D is generally meant to prepare a company for the “next generation”, be it the next generation of products, services, customers, competitors, partners or employees. However, how does the management and measurement in the R&D function differ from in the other internal functions? The nature of experimentation, which is inherent to the R&D function, explains this best. Researchers have to “try it out”; they do not know what the result will be and might even have little idea about the desired result and its effect on the planned result. This experimental modus operandi shows that the schools of thought fail in R&D; for example the business processes school of thought is very difficult to apply due to the nonrepetitive nature of the experiment. This uncertainty makes it difficult to plan, manage and measure the activities. The financing school of thought is hard to apply to R&D in the ICT sector. This is because outputs lack tangibility and perceptions about their monetary value are fuzzy. The quantity (of outputs) as such, which is often an important criterion for other departments, is of little importance for assessing R&D. A pre-condition for calculation of profitability, which sets inputs in relation with outputs, is the quantifiability of the outputs. Obstacles to the quantifiability of R&D outputs are reported in the literature: • Mapping difficulties: although expenditure on R&D is directly linked to future earnings, it is practically impossible illustrate the detail of how each Euro spent on R&D contributes to the company’s income. • Measuring the output: an appropriate methodology for measuring and evaluating R&D output does not yet exist. The output of an R&D project may be the development in the form of hardware and software, of a patent or an engineering drawing. An R&D output could also be the knowledge that a desired solution is not feasible. This output cannot always be assigned to a subsequent product. • Periodical accounting: the typical timing of the R&D projects’ results means a delay between the expense and income dates, and within the classical, periodbased cost accounting system, no direct revenue can be assigned to current expenditure. • Linkages with other functional areas: the comparison of the costs of an R&D project and the associated cash flows of the product only allows limited statements about the value-added contribution of R&D. Success within the value-added R&D phase is a necessary, but not sufficient condition, for business success. Assigning economic success to indirect operational areas is an unsolved problem in industrial economics.

Another characteristic of R&D is that R&D is “often a shared work for which it is quite difficult to assign a meaningful output to any one individual”. For over 90 years, practitioners and academics have devoted their thoughts to the matter of measurement of work performance. The question of measuring R&D productivity remains very controversial. In 1988, Brown and Svenson found that the major problem of R&D performance measurement is its acceptance. In their study, they state some of the reasons why assessment of R&D is difficult: • Many scientists and engineers think that it is impossible to effectively measure R&D productivity. In fact, the very fact of measurement is thought to discourage creativity and motivation among high-level professionals. • Many feel that management should just have faith that R&D is a good investment without trying to measure it. • Scientists and engineers feel negatively about measuring R&D performance for fearthatsuchsystemsmayexposetheirowninadequaciesandlackofproductivity. • Many attempts to measure R&D performance have resulted in dismal failure. Thesefailureshaveledmanytobelievethatallmeasurementsystemsdon’twork. Kerssens-van Drongelen states three major problems of R&D measurement: • It is difficult to accurately isolate R&D’s contribution to company performance from the other business activities because it is always the intertwined efforts that eventually result in outcomes in the marketplace. • There is a problem of matching specific R&D inputs (in terms of money or manhours) and intermediate outputs (research findings, new technologies, materials, etc.) with final outputs (including new or improved products and processes). • The time lag between R&D efforts and their pay-offs in the marketplace represent the third major problem. Previous approaches to performance management in R&D have a common defect: they deal only with selected aspects while trying to assess the performance of R&D. A similar observation is: “None of the models fully succeed in encapsulating R&D performance as none of them take all relevant factors into consideration”. Kerssens-van Drongelen also notes that there is no confirmed theoretical model for a performance measurement system. The question of the characteristics of the various dimensions of performance measurement is especially important in order to understand all of the influencing factors. Questions such as, “what is the measured object?”, “what kinds of measurement levels are  there?” and “what are the (content) areas for measurement?” are important and have to be clarified before going further in setting up a performance measurement system and deciding on performance measures. A widespread view is that process improvements are the only levers for increasing performance.

An attempt to summarize the different dimensions of R&D performance analysis has been undertaken recently by Ojanen and Vuola (2006). The authors present five basic dimensions of R&D performance analysis.  The first dimension addresses different measurement perspectives on strategy and objectives. Ojanen and Vuola list the following possible stakeholder groups: customer, internal, financial shareholders, other stakeholders, learning, and others.  The second dimension represents the purpose of R&D performance analysis. Strategic control, justification of existence, benchmarking, resource allocation, development of activities/problem areas, motivation, rewarding, and others are assigned to this category. The third dimension distinguishes different organizational levels of R&D performance analysis. The following levels have been identified: industry, network, company SBU/department, process, project, team, and individual. The type of R&D is the fourth dimension suggested. A distinction has been made between basic research, exploratory research, applied research, product development, and product improvements (incremental).19The last dimension is focused on the phase of the R&D process to be measured. Input, in-process, output and outcome are the proposed characteristics for this dimension.

This set of dimensions is one possible way to structure an overview of the different measurement perspectives. However, these examples are sometimes misarranged. For example, the “learning” of the first dimension implies that a second dimension has the purpose of measurement. The dimensions-table concept presented by Ojanen and Vuola helps to structure the multi-dimensional, multicriteria and multi-person task of measure selection, which is often difficult to execute effectively. Specifically, the understanding and recognition of the combinations of correct measurement dimensions are of value in selecting the appropriate R&D indicators.