Analysis will be based on administrative data collected by CONAFE and SEP, and the national standardized tests, to which we have full access. We have the main indicators for all compensatory schools in Mexico going back to at least 1995. We can link these data sources to the national school census (using a unique school identifier) and to the Mexican national census (using geographical identifiers) and, therefore, obtain more information at the school and community level. Thanks to the school identifier, we can also find out whether there are other educational programs present in the school, how long they have been operating, and what their current prevalence is.
In addition, four surveys have been designed. The first three are intended to measure the process outcome variables described in Section 2 above. These surveys consist of variations of a main questionnaire on: parents’ participation and involvement in school matters, perceptions on school management and on school agents’ effort and performance, school agents’ attitudes towards student learning, and the level of cooperation amongst school agents in school-related activities. Each version of the questionnaire is addressed to a different school agent: the school principal, members of the parents’ association and teachers.
The fourth questionnaire aims to collect detailed data on student characteristics and their environment. It is contemplated that both a teacher and an enumerator will help the students with any difficulties they might encounter and will assess that certain key variables (in particular the student identification number) are correct. The questionnaire includes standard measures of socioeconomic status (household size and composition, income, household assets, parental education and occupation) and some more innovative questions on the study habits, students’ impressions of the school, their teachers and the way they teach, and other behavioral questions related to hobbies and habits.
Both the process and context questionnaires will be collected at the end of the school year (June) together with the achievement test data. Moreover, measuring outcomes at the end of the school year (rather than at the start of the following academic year) is likely to minimize the potential for recall bias. A baseline round of questionnaires were fielded in October and November 2007.
6*Work Plan and Dates
We will continue to work with CONAFE to redesign the follow up questionnaires – as seems appropriate from the baseline data collection, and to help them collect and systematize the data. We will use the information collected to assess the validity of the randomization, and write a detailed evaluation report on the effects of the program on all the outcome variables aforementioned.
We propose the following work plan and evaluation timing:
July 2007: Collection of baseline school characteristics and test score data
July 2007: Selection and randomization of evaluation sample
October-November 2007: Collection of baseline data on processes
November 2007: Treatment schools receive benefits (first round)
June 2008: Collection of the first follow up of the following data:
November 2008: First draft assessing validity of the randomization and some preliminary descriptive statistics on the data gathered
November 2008: Schools receive benefits (second round)
June 2009: Collection of second follow up of questionnaires detailed above
November 2009: 2nd report with preliminary results from 1st & 2nd follow ups
November 2009: Schools receive benefits (third round)
March 2010: First complete report with results from 1st and 2nd follow ups
June 2010: Collection of 3rd and last follow up of data detailed above
November 2010: First complete draft updated with results from last follow up
February 2011: Final report completed
More succinctly, the proposed time frame by fiscal year is:
July 2007: baseline
12 month follow up
24 month follow up
36 month follow up
AGEs experimental intervention starts
November 2008: preliminary report
November 2009: draft report
7*Research Team and Organization
The principal investigator, Harry Anthony Patrinos, Lead Education Economist in the Human Development hub (HDNED), has been working on education in Mexico for the last four years, has studied the education system extensively, and has worked on the previous evaluation of the AGEs program and the urban-based school-based management program, PEC. He will manage the grant through his unit. The team includes four individual consultants: Paul Gertler, Eduardo Rodriguez-Oreggia, Stefan Metzger and Manuel Felix. Paul Gertler is the Li Ka Shing Distinguished Professor at the University of California at Berkley and former Human Development Chief Economist at the World Bank. He has extensive experience in managing large-scale prospective evaluations and has worked in Mexico on the effect of Oportunidades on child development and in education on the previous evaluation of the AGEs. Eduardo Rodriguez-Oreggia is currently Research Coordinator at the Institute for Research on Sustainable Development and Social Equality, at the Universidad Iberoamericana, Mexico City. He received his PhD in Planning and Economics from the London School of Economics. He has extensive knowledge of the education system and labor market in Mexico, and managed several research projects. Manuel Felix is the Executive Director of Investing in Education Foundation. He previously worked for six years with the World Bank Group where he held a variety of advisory assignments and worked with the office of the Chief Economist. Stefan Metzger is a Mexico-based consultant, with experience in managing large data sets, collecting information, data analysis, and is involved in Colima’s randomized PEC experiment. We will collaborate with Marta Rubio-Codina, who is also on the Colima team, and whose valuable assistance we relied on to set up the current experiment. The team will be joined by CONAFE, namely: Dr. Arturo Sáenz Ferral (Director); Lucero Nava Bolaños (Director of Compensatory Programs) and her team: Dr. Dolores Ramírez Vargas, Georgina Quintanilla Cerda, Rafaela Merecías Sánchez, María Anjelica Santiago Antonio and Teresa Nateras Valdez.
The longitudinal data will be analyzed by Eduardo Rodriguez-Oreggia, in consultation with Harry Patrinos and Paul Gertler. Interpretation of the results and relating them to the AGEs project will be handled by Harry Patrinos and Paul Gertler. The survey results will be analyzed by Eduardo Rodriguez-Oreggia, who will also support CONAFE in restyling and applying the survey instruments, and in any other data collection and data processing issues that may arise. Harry Patrinos will do the overall write-up; he will be assisted by Stefan Metzger. Patrinos will be responsible for supervising the satisfactory development of the project and reporting to CONAFE at all stages. Paul Gertler will offer overall guidance and comments on the interim results, the development and application of rigorous analytical methods, and the presentation of results. Manuel Felix will manage the supervision of beneficiary schools and relations with the private sector donors.
The final report will be disseminated electronically on the World Bank website, including the Human Development Network’s economics of education page. Research results will also be disseminated through seminars and conferences in Mexico, the United States, economics conferences and World Bank events. Lessons emerging from the study will guide policy on best educational practices to achieve higher education quality. Results will be used in Bank staff and client training. A revised version of the report will be presented at academic conferences and will be submitted for publication in an economics peer-reviewed journal.