Independent progress report
Download 0.55 Mb.
|
- Navigate this page:
- Evaluation Team
- Evaluation Findings
Next, and in discussion with Program staff in Fiji, we selected the areas for focus in the evaluation and agreed any scope limitations. During this discussion, we clarified our understanding of the Program’s theory of change.The main methods of data collection during the evaluation were secondary documentation and data review and interviews with key respondents. Respondents were selected largely by the Program from the organisations and individuals who have participated in the Program. But as far as possible, we attempted to meet key informants who had not been involved in the Program, to test and validate the information provided by Program participants.To guide the interviews, we developed a semi-structured questionnaire covering: respondents’ definition of ‘leadership for developmental change’; before and after comparison of any changes experienced at a personal, organisational and network/coalition level; respondents’ explanation of the changes identified; and their views on the Program’s contribution (to date and in the future).To assist analysis we developed an ‘evidence matrix’ as a tool to help marshal the data collected during fieldwork against each of the respective questions posed in the terms of reference. The matrix distinguished between evidence of positive Program effects, areas of weakness and suggestions for improvement. In doing this, we weighed the relative strength of the different pieces of evidence we had obtained.Finally, in order to test our preliminary conclusions, we held a feedback session with Program staff in Tonga at the end of fieldwork.Evaluation Team
Table 1: Summary of Program engagements/events1
Notes: 1 - This table does not include finance and management personnel from partner organisations trained with Program support (60), or those who they themselves have since trained (over 150). Nor does it cover training provided across 15 Pacific Island countries by UN Women with Program support. 2 - Secretariat of the Pacific Community (SPC); Pacific Islands Forum Secretariat (PIFS) Box 1: Previous Program reviews A cost-effectiveness study of the Program was conducted by Grey Advantage for AusAID in 2011. The study surveyed all 13 Program partners (22 responses – 65% response rate) and asked respondents to rate both satisfaction with and importance of different types of support provided by the Program. No element of Program support was rated lower than “important”, while across the board, respondents were on average (mode) “highly satisfied” with the Program. External monitoring from 2009 until 2010 included regular assessment of feedback from regional and (latterly) national partners. Responses appear to have been consistently positive. Reporting feedback to the first Leadership Convention, the M&E adviser noted in June 2009 that the event “was clearly significant for many partners ...The challenges and focus [were] relevant to people... [and] the long term process of building trust and openness among partners provided the right basis for its relevance to participants” Program monitoring and review reports suggest high levels of satisfaction among individuals with the support and events sponsored by the Program (see Box 1). In addition, events organised by the Program have been well-attended by at times very eminent people from across the region. The fact that the Program has been able to secure this level of buy-in arguably indicates a high degree of credibility and relevance.
|