 | Reprinted with permission from Trends in Cell Biology, 1998, Vol. 8, August, p. 335. |
Economic realities have put constraints on budgets everywhere. In science and technology sectors, these limits, coupled with ever greater numbers of researchers, have led to increased competition, and demands for accountability and relevance in research. Scientists and the government are obliged to find ways to evaluate and justify research expenditures. The American Association for the Advancement of Science (AAAS) participates actively in a range of science-policy areas. Their 23rd Annual Science and Technology Policy Colloquium [1] discussed the details of the 1999 research budget, annual projections, the changing roles of scientists in government, and an important piece of legislation that has come into effect recently.
The GPRA
In 1993, the U.S. Congress passed the Government Performance and Results Act (GPRA) to improve coordination among research agencies, Congress, and the White House, and to develop a more effective allocation process for research funds. It was designed to add credibility to the distribution of research dollars by removing historical and political factors and making it entirely performance based.
GPRA attempts to do at the research-agency level what grant study sections do at the individual-investigator level. Agencies receiving federal research funds are required to develop a five-year strategic plan summarizing their aims. The plan must include specific annual goals, a discussion of how these goals relate to the long-term plan and how the aims will be implemented. Congress can then score agency proposals and rank them for funding priority. The most controversial part of GPRA, and of research assessment in general, is the requirement for methods to measure performance in research.
There is concern that the larger goals of science will be lost to quantitative measures of progress. It is not obvious how to evaluate or forecast progress in basic research on a year-by-year basis. GPRA does make some provisions, but no excuses, for the unpredictability of research. As Joshua Gotbaum of the Office of Management and the Budget summarized, "Research is different, but it is not exempt."
Assessment Elsewhere
The unique nature of research is a problem faced by funding-allocation systems worldwide. The UK has been using performance assessment to prioritize funding for many years. The process has gone through many changes and is still maturing. Ben Martin of the Science Policy Research Unit, University of Sussex, United Kingdom, discussed the present system and its history to show that not only could an assessment program be developed, but it could succeed. He concluded by suggesting further improvements, most notably calling for a greater involvement of the research community in the evaluation process.
Performance assessment is here to stay, and the scientific community has an important role to play in ensuring that it is successful in allocating funds appropriately. One issue that remains unsettled is risk assessment. Failure is an integral part of the scientific process; can it be incorporated productively into strategic planning? "Playing it safe" could result in research that progresses but lacks strong leadership and innovation. The assessment process needs to include a method of measuring risk, while still underlining its importance in the scientific endeavor. The discussions at the meeting highlighted that performance evaluation in research is still developing and that, as more factors are added to the equation, the system will hopefully evolve into one that meets the needs of science, funding agencies and the taxpaying public.
Amy Muhlberg is a graduate student at the Scripps Research Institute.
Andrzej Krauze is an illustrator, poster maker, cartoonist, and painter who illustrates regularly for HMS Beagle, The Guardian, The Sunday Telegraph, Bookseller, and New Statesman.
Endlinks
Allocating Federal Funds for Science and Technology - addresses the criteria that should be used in the allocation of funds to research and development activities. A joint study from the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
Assessing Fundamental Science - provides general principles and goals for the assessment process in science. From the National Science and Technology Council.
Continuity and Change in Science and Technology Policy - an excerpt from Daniel Lee Kleinman's "Politics on the Endless Frontier: Postwar Research Policy in the United States" (Duke University Press, 1995). Maintained by the Indiana University Institute for Advanced Study.
Science Funding - an HMS Beagle Cutting Edge Debate.