Source: Pacific Leadership Program data Evaluation Objectives and Questions

4.

1. Evaluation Objectives and Questions

   1. The Australian aid program’s Performance Assessment and Evaluation Policy requires aid activities that have been running for four or more years to be independently evaluated during implementation. The purpose of these on-going evaluations is threefold:

• assess progress against objectives;

• improve implementation quality; and/or

• inform the design of any follow-on phases or new activities

1. The current phase of the Pacific Leadership Program is due to complete in June 2013. Initial work for the design of Phase 3 coincided with the evaluation exercise. This Independent Progress Report is expected to inform the next phase of Australia’s support to leadership in the Pacific.

2. The terms of reference (see appendix 3) directed the evaluation team to focus on six main issues:

1. 
   the extent to which the Program has helped strengthen individual leaders’ capacity;
   
2. 
   the extent to which the Program has helped to strengthen leading organisations in target sectors;
   
3. 
   the extent to which the Program has supported coalitions of leaders to exercise leadership and enable change;
   
4. 
   the adequacy of the Program’s monitoring and evaluation and learning processes;
   
5. 
   how well the Program has learned from evidence and experience to evolve to meet the leadership challenges facing the Pacific; and
   
6. 
   how well the Program has communicated with and complemented AusAID bilateral, regional and international programs.
   

5.In addition, the terms of reference asked for a cursory assessment of the Program against the DAC criteria of relevance, effectiveness, efficiency, impact, sustainability, and the AusAID criteria of monitoring and evaluation, gender equality and analysis and learning. Some of these criteria are implicitly or explicitly addressed by the main issues raised in the TOR, with perhaps the exception of gender equality, which we considered specifically.

3. Evaluation Scope and Methods

1. This evaluation covers the period from the Program’s inception in 2008 to March 2012 (the start of the evaluation). As a progress evaluation, we have not attempted to assess Program impact formally; in considering Program effectiveness, however, we do provide insights on the effects of the Program, based on the use of informal techniques. Nor have we examined Program efficiency in any detail but instead have taken assurance from the latest QAI report and a review of Program cost-effectiveness conducted in 2011 by Grey Advantage. This decision reflects both the direction to the review provided by the TOR and the time available.

2. We have explicitly excluded a number of activities from the evaluation that the Program funds but which have not been core to its work. These include: the Greg Urwin Awards, Emerging Pacific Leaders Dialogue, Emerging Pacific Women’s Leadership Program, support for the Centre for Democratic Institutions and the Gender Equality in Political Governance (GEPG) Program implemented by UN Women. The latter was subject to a separate evaluation at the time of our assessment. Collectively, these elements comprise around 23% of total Program expenditure to date.

3. We made field visits to Fiji (5 days) with regional partners and stakeholders and in two of the Program’s four target countries: Vanuatu (3 days) and Tonga (5 days). The Program selected these countries, on the grounds that they provide good coverage of the range of activities and experiences of the Program to date.

4. Our only concern with this selection was the omission of Solomon Islands, which is the largest of the Program’s target countries in expenditure terms. As a result, we supplemented the design with telephone interviews with the largest partners in Solomon Islands (by expenditure): the Solomon Islands Development Trust, the Solomon Islands Women in Business Association and YWCA.