Strengthening leadership is important but challenging, as is demonstrating the effectiveness of efforts. A Program of this sort could be quickly consigned to the margins of relevance if it failed to establish credibility with stakeholders. This has clearly not been the case. More generally, leadership development is often treated as a long-term investment,, leaving questions of effectiveness in the shorter term unanswered. But the evidence indicates that the Program has to date contributed to some important developments nationally and regionally. Moreover, our findings suggest that the methods being deployed to engage, strengthen and challenge current and emerging leaders and leading organisations are highly relevant and achieving a good level of effectiveness.
Preliminary work on the design of the phase 3 of the Program coincided with our evaluation. During the course of our fieldwork, we identified a number of areas where we believe there is scope to strengthen the Program. We offer the following conclusions and recommendations to inform the deliberations about the next phase.
The Program should develop a simpler overall M&E framework - building on the draft framework and theory of change, and this evaluation. The potential value of clarifying Program expectations and strategy and tracking change more systematically has been a consistent message through our evaluation. We have also suggested a number of tools that the Program could adapt and apply – e.g. organisational capacity assessments, partnership maturity assessments, social network analysis. We see three main benefits of this: intelligent monitoring against explicit expectations supports learning and the refinement of ‘theory’; application of consistent methods can guide decision-making and provide insights not provided by reflective learning processes alone; and use of more formal assessment tools can help the Program communicate progress more effectively to external stakeholders.
In recommending this, however, we want to clarify two possible misunderstandings. The view was expressed to us that more explicit strategy and more systematic assessment techniques risked imposing a rigid, prescriptive implementation approach on the Program. We do not believe the two are synonymous. Greater rigour in the way the Program approaches its own learning does not require a blueprint approach to programming and can support the logic of what some have called development entrepreneurship.22
Second, any M&E framework needs to be proportionate. We accept that in a Program of this sort there will be areas that are more ‘complex’– either because of their inherent unpredictability or because too little is known about the change mechanisms to enable effective monitoring based on predetermined indicators. In such cases, deeper, evaluative processes will be required. However, there are also those more ‘stable’ or straightforward areas, which are amenable to the routine collection of data. Distinguishing between the two types is the first step in developing a simpler, more effective framework.
To guide the detailed design of the framework, the Program should further define its theory of change. This will help clarify core assumptions underpinning Program activities, which in turn will help frame the key questions for M&E and provide the necessary focus. Paragraph 2.4.18 above elaborates a basic structure to assist with framework design.
In line with the above, we also recommend specifically that the current M&E framework for the Tonga Secondary Schools Leadership Program (TSSLP) include the explicit objective and means to allow longer-term impact to be assessed. We recognise that this will need to be done sensitively, requiring creative approaches to involve key stakeholders, and may require some compromise in terms of ‘purist’ rigour; but TSSLP offers an important opportunity to provide more concrete evidence of the contribution of leadership to developmental change, which should not be missed. It may also contribute to current efforts within AusAID to assess the impact of education sector support more rigorously. The Program may require specialist research support to implement this recommendation.
The Program should give greater priority to the objective of informing practice in the broader Australian aid program. It is well placed to do this, by building on its strong processes of reflection and learning, and the investment it has recently made in research and communication staff, to better document and share experience. Its evolving approach to Partnership, and what is being learnt about leadership in the Pacific are two obvious initial areas of focus. We also see value in this for the Program team, in terms of enabling them to communicate lessons and experience (e.g. on partnership, finance training, and organisational development) for the benefit of program and for others. It also a means to mitigate risks associated with breaks in continuity if and when key personnel move on.
