Economic impact assessments

Some 20-30 years ago, development projects were largely the product of engineers and economists (Rashid, 1998). A combination of technical criteria and internal rates of return determined project design. Whilst the objective was poverty alleviation, the main focus was essentially on increase in production, particularly in the agricultural sector. Economic impact assessments generally estimate the economic benefits produced as a result of the interventions under study (Cost-Benefit Analysis, Rates of Return). The methods employed in economic evaluations often ignored social, environmental and institutional effects.

Since the 1980’s, social and institutional aspects did get a toehold in the thinking about project design and evaluation. Gender and environmental issues started to get wider attention as from the mid-eighties. Social and environmental impact assessments are concerned with the broader effects of a research project or activity on society and the natural environment. Such assessments go beyond the examination of economic returns and look at the effects on such things as cash flow, labour, or health. Especially the introduction of environmental and gender impact assessment studies has pulled the assessment of projects away from the sphere of economic appraisal only.

With increasing concerns for social and environmental issues, several specific impact assessment-screening tools came to co-exist. In addition to CBA and EIA we can mention: Social Impact Assessment (SIA), Gender Assessment Study (GAS) and Poverty Assessment (PA). This gave rise to a rather poorly co-ordinated aspect-by-aspect approach that has several weaknesses: ignoring interdependencies between sectors, misjudgement of impacts and oversight of valuable alternative solutions. This has led towards increased interest for crosscutting issues and a movement towards more integration. However, many conceptual problems remain for integrating thematic assessments (Scholten and Post, 1998).

Assessing the effects of innovations is complex. Several difficulties can be encountered:

• 
  First, there are measuring problems. It is quite difficult to measure changes in yield, production, nutritional status, and erosion, and it requires costly fieldwork and analysis. Agricultural research organisations often lack the personnel and operating funds needed for this, especially when several growing seasons are required for changes to be measured in most yield and production systems. Trends in dry land agriculture, for example, cannot be measured in a few cropping seasons.
  
• 
  Attribution problems. Second, even where a change can be measured, it is extremely difficult to attribute it to specific research activities. For example, in an area where potato yields or milk production has increased, how can the contribution of research versus that of extension, credit programs, and improvements in market conditions be estimated?
  
• 
  Time lags. Third, research managers, policymakers, donors, and the public all tend to be impatient and to want impact estimates when research is still underway or has just recently been completed. This is neither realistic nor possible. There is often a considerable time lag, first from the start of research to the release of a technology and from the start of dissemination to the impact at farm level.
  
• 
  Assumptions and simplifications. Impact assessments generally employ scientific methods drawn from economics and the social sciences and often use indirect indicators of impact because the effects of technology on farm-level production, nutritional status, pollution, and the like cannot be directly measured. To cope with this problem, production-function models are often used to estimate the effects of research or technology on production, incomes, and associated variables. Numerous assumptions are also often made to overcome data limitations and to simplify economic models.
  
• 
  Scepticism. Research managers and policymakers tend to be sceptical of the data and methods used in impact assessment. They may also find the reports difficult to understand, interpret, and apply. This highlights the need to pay close attention to data quality, and to make special efforts to summarise the findings. It is extremely important for results and recommendations to be presented in terms that are meaningful to policymakers, managers, and scientists.
  
• 
  Decorative use. Often, the impact study is actually not more than just an administrative requirement and policy makers (already having taken their decisions) never considered to use the results anyway. This is the fate of many environmental impact studies and gender impact studies. Also, donors have thematic policies (gender, environment, culture, equity, institutional capacity) for which they require regular baseline and impact studies. In practice, the results of these studies are hardly used (Scholten and Post, 1998).
  

The future of impact assessment studies is not very clear. If the donor focus is on sector approaches, there is a possibility that thematic impact assessment studies (and the problems related to the aspect-by-aspect approach) will prevail. If the focus is on higher-level goals like poverty alleviation, the tendency towards an integrated approach to impact assessments may be strengthened.

More than ten years ago, it was already noted that “despite the sizeable research network, Tanzania research services have not been able to fulfil their role in developing appropriate technological packages for farmers” (URT, 1991: 13). The major problems are poor stakeholder involvement, weak research-extension linkages, inadequate funding, low staff motivation, insufficient output, insufficient adoptability analysis, poor co-ordination and backstopping, fragmentation and poor co-ordination of the research system.

• 
  Interpretative approach Research approach that tries to understand a situation from within, as if through the eyes of the people under study.
  
• 
  Survey approach Research approach that compares the characteristics of people showing certain behaviour (for instance adoption of innovation) with people without that behaviour. Different characteristics of these groups are supposed to give insights to the understanding of different behaviour patterns.
  

Peterson, W. and D. Horton (1997). Impact assessment. Handout 5/14/6 of ISNAR training course on research planning, monitoring and evaluation.

Presents background information on international agricultural research centers, discusses their impact on agriculture through work in plant breeding and farming methods, and assesses changes in human resources and the capacity of national organizations.

Analyzes the socio-economic impact of investments in agricultural research. Focuses on actual and potential benefits and on the social rate of return to these investments.

A collection of papers on environmental impact assessment. Includes methodological guidelines and case study reports.

An extensive literature review designed and organized to be of direct practical use to students, teachers, and practitioners responsible for organizing social impact assessment (SIA). SIA is viewed as a response to the ever-growing need for full and open information and involvement in decision-making. Summarizes views on a large number of topics related to SIA and includes numerous references.

Papers on impact assessment concepts, experiences, and results. Topics include impact of international agricultural research centers, impact of research on food quality, impact of private- and public-sector research, impact of farming systems research, an evaluation of research extension in Peru, and an assessment of the impact of research on pasture and cassava improvement in Latin America. Covers both ex ante and ex post assessments.

An overview of basic issues and concepts related to sustain-ability. Outlines research policy issues and discusses the measurement of sustainability. Useful background reading for environmental impact assessment.

Issues related to impact assessment for international agri-cultural research centers. Presents an interdisciplinary approach to the review and assessment of accomplishments.

Uses a survey of 245 homesteads as well as secondary data to assess the diffusion of hybrid maize varieties and their impact. Concludes with an assessment of macro-level benefits and costs.

A summary of economic assessments of 12 successful ACIAR project areas. The paper concludes that the benefits from these 12 project areas alone are sufficient to recoup all ACIAR project costs to date, even if the remaining 166 projects were to produce no economic benefits at all. The summary confirms the general view that agricultural re-search represents a wise investment of development-assistance funds.

Informative and useful look at estimating impact based on information collected from farmers.

An interesting example of an adoption study that analyses the dynamics of farmer-to-farmer distribution of new bean varieties. Outlines a diffusion strategy for improving small farmers’ access to new varieties, recognising that adoption is not a one-time affair and that poorer farmers have particular difficulty in receiving and keeping new cultivars.

Winpenny, J.T. (1991). Values for the environment: A guide to economic appraisal. HMSO, London.

Designed to give economists working in developing countries a summary of the state of the art on the methodology and practice of economic appraisal of environmental impact. Includes a wide range of case studies.

Innovation and diffusion

1. 
   What decision-making pathways do individuals fallow when considering whether or not to adopt an innovation? Which sources on information are important?
   
2. 
   What are the differences between people who adopt innovations quickly or slowly?
   
3. 
   How do the characteristics of innovations affect the rate of adoption?
   
4. 
   How do potential users communicate among themselves about these innovations? Who plays the important role of opinion leader in this communication process?
   
5. 
   How does an innovation diffuse through a society over time?