Defining Effective Drug Prevention Programs: What Works?

Tobler Research Associates conducted a systematic review of studies examining the effectiveness of tobacco and substance abuse prevention programs. The Robert Wood Johnson Foundation (RWJF) provided support for the study through a contract with Tobler.

Under the contract, investigators conducted three separate meta-analyses examining:

• 
  207 evaluations of school-based substance abuse prevention programs
  
• 
  99 evaluations of non-school-based programs
  
• 
  51 evaluations of family interventions that measured aggression and conduct disorder — two commonly used predictors in studies of young-adolescent substance abuse
  

• 
  Interactive programs — which foster interpersonal skills and active engagement between students and teachers — are more effective than noninteractive programs — which are lecture oriented and stress drug knowledge. Interactive programs are more effective at reducing, preventing or delaying adolescent drug use for all substances combined and for each substance individually — tobacco, alcohol and marijuana).
  
• 
  As the number of participants in an interactive program increases, the program's effectiveness decreases. Non-interactive programs are only marginally effective at any size, yet are used in 90 percent of U.S. schools.
  
• 
  Many of the reports that list "evidence-based best practices" or "research-based model programs" include programs for which evidence of effectiveness is of questionable quality or lacking altogether.
  

The following findings were reported in the Journal of Primary Prevention (summer 2000).

• 
  Interactive programs — which foster interpersonal skills and active engagement between students and teachers — are more effective than noninteractive programs, which are lecture oriented and stress drug knowledge. Interactive programs are more effective at reducing, preventing or delaying adolescent drug use for all substances combined and for each substance individually — that is, tobacco, alcohol and marijuana).
  
• 
  The magnitude of the effectiveness of interactive programs is equivalent across age and ethnic groups and is the same for tobacco, alcohol and marijuana. These programs take universal approaches.
  
• 
  There are three distinctive types of interactive interventions, listed in descending order of effectiveness: systemwide change, comprehensive life skills, and social influences.
  
  • 
    Systemwide change programs include school-based programs that a family and/or community actively supports and programs with a more singular focus on creating a caring, supportive school environment.
    
  • 
    Comprehensive life skills programs focus on interpersonal skills development, knowledge and life skills training, including assertiveness, coping skills and setting goals. They also have a broad affective component, which focuses on the individual's sense of self and perception of the world.
    
  • 
    Social influences programs focus primarily on interpersonal skills development and include a knowledge component emphasizing refusal skills training. These programs may include a limited affective component.
    
• 
  As the number of participants in an interactive program increases, the program's effectiveness decreases, probably due to implementation failures.
  
• 
  Noninteractive programs are only marginally effective at any size, yet are employed in 90 percent of U.S. schools. Since enactment of the Drug Free School Act in 1986, virtually all U.S. schools have adopted some kind of prevention program.
  
• 
  Targeting tobacco alone, whether in an interactive or a noninteractive program, is more effective than including tobacco with alcohol, marijuana and other drugs in a general substance abuse prevention program.
  
• 
  Targeting alcohol alone, however, does not increase program effectiveness.
  
• 
  The success of interactive programs is related to program intensity (intensity is measured by the amount of time devoted to the program). The more-intensive interactive programs are more effective than those that are less intensive.
  
• 
  Devoting more time to a noninteractive program did not increase its effectiveness.
  
• 
  Many of the reports that list "evidence-based best practices" or "research-based model programs" include programs for which evidence of effectiveness is of questionable quality or lacking altogether.
  

• 
  Researchers were unable to confirm or refute their hypothesis that programs geared toward a general population are less effective with high-risk youth.
  
• 
  Due to insufficient data, the researchers could not isolate individual characteristics of programs to determine their effects on high-risk youth. The implementation of interventions targeting selected populations — that is, high-risk or nonmainstream youth — varied much more than universal school-based interventions. For example government agencies that had the power to incarcerate people who did not successfully complete the program implemented some interventions, and these programs were generally more effective than interventions not mandated by law. Researchers were uncertain which factors most accounted for the apparent effectiveness of involuntary programs.
  

Grant Results Report

From 2000 to 2002, researchers at Social Capital Development Corporation, a community development and social science research firm, analyzed outcome data from more than 200 school-based drug education/prevention programs. The analysis was designed to help the field understand which programs work with which students and how best to deliver the programs. In contrast to earlier findings, the researchers found that comprehensive life skills programs were not more effective than social influences programs at all grade levels and with all types and levels of substance use and that the difference between the effectiveness of interactive and non-interactive programs across all grade levels was not statistically significant. The Robert Wood Johnson Foundation (RWJF) supported the research with two grants, from April 200 through January 2003, totaling $299,712.

Researchers reported these findings from the first phase of their work at the RWJF-sponsored Prevention 2000 summit (ID# 039287) (see Communications):

• 
  A reanalysis of data from the 1999 Tobler study indicated that comprehensive life skills programs were not more effective than social influences programs, at all grade levels and with all types and levels of substance use.
  
  • 
    Comprehensive life skills programs appeared to work better at the elementary and senior high school levels, but only in terms of an impact on students' use of cigarettes and hard drugs. There was no significant difference in impact on students' use of marijuana or alcohol.
    
  • 
    At the middle school level, there was no difference between the effectiveness of social influences and comprehensive life skills programs for reducing the use of cigarettes, marijuana and other drugs. Social influences programs were moderately effective at reducing heavy drinking among middle school students, but ineffective at reducing alcohol use.
    

• 
  Using new modeling techniques, the overall difference between the effectiveness of interactive and non-interactive programs across all grade levels was found to be not statistically significant. Interactive programs were found to be significantly more effective than non-interactive programs at delaying, reducing, or preventing tobacco use, but not the use of other substances. When broken down by grade level, interactive programs worked better in middle school but not in high school.
  
• 
  The duration of a school prevention program may account for the large differences Tobler found between interactive and non-interactive programs. The beneficial effect of interactive programs occurs more than 18 months after pretesting, rather than a year, as Tobler reported