Assessing the Benefits of Public Research Within an Economic Framework: The Case of USDA’s Agricultural Research Service
by
Paul Heisey,
John King,
Kelly Day-Rubenstein, Dale Bucks, and Rick Welsh
Economic Research Report No. (ERR-95) 90 pp, May 2010
Agricultural research managers continually seek to identify the
benefits of research programs, both to demonstrate social and
economic impact and to prioritize future research. Tight budgets
for public agricultural research have increased the need to get the
most out of these investments. Meanwhile, shifting policy goals
have pushed public agricultural research in new directions.
Although economic analysis can provide quantitative estimates of
research benefits, it may not be amenable to all situations.
However, economic reasoning may be useful even when formal economic
methods are not used.
What Is the Issue?
USDA's Agricultural Research Service (ARS) accounts for the
majority of Federal expenditures on agricultural research. The most
common method of Federal research evaluation is peer review-widely
used throughout the ARS prioritization, planning, and evaluation
cycle. Peer review's strength is assessing scientific merit.
However, it is not well-suited for quantifying market impacts or
ultimate social benefits of research programs. Economic analysis,
by contrast, can address these benefits, but often at considerable
cost.
What Did the Study Find?
The standard techniques of economic evaluation-especially
econometric and economic surplus techniques - have usually been
applied retrospectively, and at aggregate levels of analysis. Three
case studies of ARS research-bovine quantitative genetics/genomics,
water quality/watersheds, and nutritional composition of
food-illustrate opportunities and limitations of applying existing
methods of economic evaluation to individual programs of research,
and demonstrate the value of economic reasoning even in the absence
of quantitative analysis. These case studies were selected to
capture research diverse in its nature (basic versus applied),
program scope, coordination roles, and relevance to USDA missions.
As such, the case studies are not meant to represent the full range
of issues addressed by ARS; rather, they focus on some of the more
challenging aspects of research evaluation.
Case Study Specifics
• Bovine Quantitative Genetics and Genomics. ARS
researchers conserve animal genetic resources, identify genes
related to economically important production traits in beef and
dairy cattle, and estimate the heritability of desirable genes. The
benefits of ARS research can be measured by the value of the
increased productivity engendered by the research, so standard
economic analysis is plausible. Nonetheless, allocating attribution
among the various contributors to increased productivity adds to
the complexity of the problem.
• Water Quality and Watersheds. ARS research on water
quality and watersheds examines physical and chemical properties of
water and agricultural pollutants, their impact on agricultural
production and water quality, and methods to improve agricultural
and environmental outcomes. This research generates both private
and social benefits. The benefits of research leading to the
development of specific products or services, such as irrigation
control equipment, are amenable to standard economic analysis. But
the benefits of research on factors affecting water quality are
harder to evaluate, requiring the valuation of nonmarket
goods/services and linking research with public policies and
regulations.
• Nutrient Data Laboratory. The Nutrient Data Laboratory
(NDL) provides data on the nutrient composition of foods in the
American diet. Its databases are the foundation of virtually all
public and commercial nutrient databases used in the United States
and a number of other countries. The primary social benefit of
NDL's research is improved human health, but the link between
nutrition information and health outcomes, while widely recognized,
is not well
understood. Coupled with difficulty in assigning a dollar value to
health outcomes, these measurement and attribution problems make
standard cost/benefit analysis problematic.
Economic reasoning can provide qualitative analysis even when
quantitative estimates of benefits are intractable. For instance,
market failures and the presence of significant cross-State
application of research findings may lead the private sector and
State-funded institutions to underinvest in economically
justifiable research. All of the case studies show that ARS
research provides scientific results with few substitutes. Because
of their "public goods" nature, the benefits of these research
programs would be difficult to replicate by non-Federal research
efforts. Thus, qualitative analysis can provide a clear indication
of the public goods characteristics of a research program, even if
it cannot rank multiple projects or programs, all of which
demonstrably address public goods.
Furthermore, interviews with stakeholders such as food
processors, natural resource managers, universities, other Federal
agencies, and international research institutions suggest that ARS
research facilitates numerous linkages between peers, contributors,
and users. While it is difficult to put a dollar estimate on these
benefits, interest on the part of a diverse group of stakeholders
may indicate a broad set of benefits.
Finally, public research often aims to enhance the operations of
Federal regulatory agencies and strengthen the scientific basis of
government policies. Assessing research programs aimed at improving
government regulations generally exceeds the scope of economic
analysis because they work through the political process. However,
economic reasoning is useful for tracing the demand for and
performance of mission-related research. In these cases, ARS
research often contributed to the regulatory and policy functions
of the Federal Government in ways that other public sector research
institutions did not. For example, with passage of the Clean Water
Act, ARS adapted its soil movement models to examine the effects of
sedimentation on downstream water quality. The findings of the
three case studies provide additional insights about important
issues in assessing research benefits.
• Basic research and nonmarket goods. In Federal
research evaluation, benefit-cost analysis has been used primarily
for Federal research programs that produce specific, near-market
technologies. While some ARS research produces near-market
technologies, two of the case studies (and much of the third)
involved research producing significant nonmarket benefits or basic
research with no immediate market applicability.
• Attribution. Research builds on previous findings,
and numerous related efforts are often being performed by other
institutions. Attributing benefits to one group of researchers is
imprecise at best. All three case studies featured significant
numbers of different researchers, with numerous horizontal and
vertical linkages.
How Was the Study Conducted?
This report focuses on the feasibility of using economic and
other evaluation methods to value ARS research, rather than the
estimation of quantitative values. This study combined two
principal methods-detailed literature reviews, and in-depth
qualitative interviews with ARS research administrators,
scientists, and stakeholders. The interviews used an interview
guide approach that featured structured but open-ended questions,
with information-rich respondents queried until answers confirmed
working conclusions rather than revealing new important topics.