Review of the dfat ending Violence Against Women (evaw) Program in Afghanistan
Download 392.87 Kb.
|
- Navigate this page:
- ME System and Use of Monitoring Data
- Performance Assessment Framework
- 3.4 Efficiency
- Coordination
- Other Donors and Programs
3.3 Monitoring and EvaluationIn assessing the monitoring and evaluation of the program, the review focuses on the M&E systems and reporting mechanisms in place, the use of monitoring data, and the overall appropriateness of the PAF for the program. Additionally, the review addressed the utility of the current M&E system in terms of facilitating a shared measurement system and assessing progress at a programmatic level. M&E System and Use of Monitoring DataThe current M&E system for the project is currently highly output-focused, and a number of indicators in the Performance Assessment Framework (PAF)—both quantitative and qualitative—are not supported by routine data collection to inform them. Overall, to date the monitoring data that has been communicated in reporting from IPs is adequately documenting progress in terms of outputs, but does not provide sufficient information to monitor whether these outputs are contributing to outcomes as planned. Each partner utilizes a different monitoring and reporting format, which are similarly activity-oriented, and do not necessarily require the collection or reporting of data according to indicators in the PAF. This lack of alignment in M&E and reporting systems among partners for the program presents a challenge in terms of an effective shared measurement system. The challenge was noted among partners in terms of the differing modalities each organization uses for monitoring, and the need to ensure that systems are aligned with each partner’s existing monitoring procedures and infrastructure. However, there is room for flexibility in terms of the means collecting data while still aligning the data collected with the overall PAF. The lack of data to reflect progress at the outcome level furthermore limits partners’ and DFAT’s ability to utilize monitoring data in decision-making. As explained by the partners and Core Steering Group (CSG) participants, the CSG is primarily utilized as a forum for partners to report on their activities, with limited engagement between the partners in terms of reviewing progress at the outcome level. It was noted that its utility could be improved by utilizing the platform to review not only progress in terms of activities and outputs, but to collectively review progress at the outcome and intermediate outcome levels. Performance Assessment FrameworkThe overall PAF for the program needs refinement and realignment to reflect progress and implementation experience to date. Some outcomes are not fully aligned with the overall goal of the program, and the current PAF does not adequately reflect the collective impact of the projects under the program. Outcome two focuses on the justice sector, which is not a central focus of partners’ activities, whereas the police sector is not specifically reflected in the PAF. The peace process component under outcome three is not adequately aligned with the program or its overall goal, and is largely irrelevant to the rest of the PAF and the wider EVAW program. The PAF should be updated to incorporate AIHRC and AWN activities under the program, as their activities’ contributions toward the achievement of program outcomes and goals has not been incorporated into the measurement system under the PAF since the addition of these components in the second phase of the program. Additionally, capacity building of key government stakeholders is not adequately reflected in the indicators of the PAF, which largely reflect the trainings delivered but do not attempt to capture the institutionalization of change within the relevant stakeholder institutions. Currently, each of the three outcomes under the PAF is heavily aligned with one specific partner and project, though various activities from each partner align with each outcome. This was also evident in the Strategic Learning Debrief, where each partner felt that their activities were most relevant to one of the outcomes, which evidenced a lack of considered reflection on how the program activities are contributing to multiple aspects of the program’s planned outcomes. The PAF and M&E system for the program are also limited in terms of attribution, with several indicators measuring macro-level changes that align with DFAT activities but also a wide range of other donors and programming engaging in the sector and without adequate consideration to geographic location of the respective programs. At this stage in the program, it is necessary to revisit the PAF collectively with all implementing partners to follow on this midterm review and collectively revise the intermediate outcomes accordingly, then revisit the indicators under each component of the project. Furthermore, it is important to review and revise the indicators in the context of the monitoring data that has been collected by partners to date. Revising the PAF through a facilitated participatory learning process with face to face interaction between all partners would likely support the development of a more relevant and coherent PAF. 3.4 EfficiencyIn the scope of this review, efficiency is largely considered in terms of governance and management arrangements and the coordination and coherence of the program with the government and other donors and programming in the EVAW sector. Governance and Management ArrangementsImplementing partners largely expressed satisfaction with the governance and management arrangement of the program under DFAT, particularly noting the flexibility and responsiveness of DFAT as a donor. All partners appreciated the level of support they receive in terms of no-cost extensions, changing and expansion of activities and target areas, amendments and revision of project documents related to external challenges such as currency depreciation, and addressing some cases of of under-expending by local partners. Partners felt that the CSG—a semi-annual meeting among DFAT, implementing partners, and DFID—has been a useful platform for coordination and development of relationships among partners. The forum has helped partners to proactively address potential duplication, such as the support planned by UN Women and TAF to EVAW Commissions that was subsequently removed from TAF activities. The forum also provides an opportunity for coordination on a geographic level, where partners regularly review which activities are being implemented in which provinces and districts to facilitate coordination and joint efforts where programs overlap geographically. Through this platform, partners with similar activities and direction of work are encouraged to coordinate and share training manuals and other project materials produced such as SOPs, needs assessments and evaluation reports, strategy documents, and other relevant resources that can be utilized by other partners. The review identified different levels of understanding of the CSG by different partners. While some view it as a forum for reporting activities, others are interested to have it as a platform to discuss issues at the outcome level and facilitate and more targeted lessons learned and best practices sharing and utilization. While the CSG is an important forum for the oversight role of DFAT, the forum should be also owned by partners through their proactive participation, agenda setting, and utilization of each other’s lessons learned and best practices. A specification of the CSG objectives and functionality could be improved by developing a ToR giving specific responsibilities to partners and encouraging their meaningful and proactive participation. After the rapid review in October 2014, it was agreed that the CSG plenary would be extended to include local partners. However, this has proved impractical to date, with each CSG since then requiring specific information or topics to be addressed and making it not possible to invite local partners to participate as well. All local partners showed a huge interest in direct communication with DFAT periodically which they felt would allow them to raise management and implementation issues to both direct and indirect funders. The reactions of some implementing partners to involving the local partners was mixed in terms of its perceived potential impact on the implementation of the program. However, the review concludes that it could provide an important forum for making linkages throughout the program from the donor down to those directly implementing on the ground, and could be a valuable opportunity for supporting the further capacity development of local partners. CoordinationGovernmentOverall the program is consistent with national and international frameworks including the National Action Plan for the Women of Afghanistan (NAPWA), National Priority Programs (NPPs), the Afghanistan Millennium Development Goals (AMDGs), CEDAW, UNSCR 1325, and the Self-Reliance through Mutual Accountability Framework (SMAF) and Tokyo Mutual Accountability Framework (TMAF). Furthermore, consultations with government stakeholders generally evidenced an alignment of program activities with government objectives and priorities. The review verified the completion of activities reported under effectiveness through consultations with various stakeholders, and coordination and dialogue between implementing partners and key sector stakeholders was largely evident at the central level across all projects. However, there were limitations noted in the coordination of the UNFPA program with the wider police sector stakeholders, where effective coordination was noted with the MoI Gender Unit, but was less apparent with the Police Academy and the Ministry of Justice. Coordination of the program with key government stakeholders has been formalized through a series of MoUs that support the projects. TAF has signed MoUs with the AGO and MoWA, and these institutions have assigned an official representative to monitor TAF activities that serve as a focal point for coordination between TAF and the institutions. According to TAF, this mechanism has proven effective, with any issues encountered at the provincial level relayed to the central level focal points for the project who were able to address the issue. Similarly, UNFPA has signed an MoU with MoI, and HEWAD also reported having an MoU in place with MoJ in relation to the project. However, HEWAD noted that they were unable to secure a MoU with the Supreme Court due to a lack of high level engagement. UN Women has implemented their programming through a joint work plan with MoWA, and UNFPA noted that a similar mechanism would be useful to develop with MoI. UN Women has implemented their programming through a joint work plan with MoWA,. The findings of the review also suggest less effective coordination with government stakeholders at the provincial level, such as the AGO and DoWA. The local partners were not well-known to stakeholders meant to be receiving support and participating in programming at the provincial level. The limited awareness of select government stakeholders at the provincial level would suggest the need for further focus on coordination and ensuring activities are effectively coordinated with all key government stakeholders. In terms of local partners’ engagement with provincial office of the government, issues of insecurity, corruption, and demands and expectation from individuals within key stakeholder institution was noted as impeding smooth coordination and sometimes requiring intervention from stakeholders at the central level and support from their donor to address the issue. Though the challenges of coordination at this level were noted, further efforts in this regard are critical to the sustainability of the activities and their outcomes.
However, coordination—particularly among donors—was noted as a limitation across the EVAW sector. Various coordination mechanisms related to gender are in place, but coordination specifically on EVAW support is generally ad hoc. The Gender Donor Coordination Group and ARTF gender meetings provide an opportunity for donor coordination on gender issues, but this encompasses the wide range of gender-related interventions and support. The EVAW Special Fund (EVAW SF) Advisory Board also provides an opportunity for coordination, but it is largely a forum for UN Women donors, and though various groups attend the meeting there are fewer donors present. The GBV Sub-Cluster (GBV SC) was started in 2009 and is chaired by UNFPA, and AIHRC has signed a MoU with UNFPA on GBV SC coordination group and co-chairs the monthly meetings. In 2015 the platform was revised and the ToR aligned with its intended purpose, where the GBV SC is meant to coordinate during emergencies according to interagency guidelines. This can be a challenge in Afghanistan, which is in a chronic state of humanitarian emergency and the distinction between humanitarian and development work on GBV can be unclear. However, EVAW sector stakeholders clarified that while the GBV SC is a useful platform for its intended purpose, it does not provide an opportunity for donor coordination in the EVAW sector. There are informal coordination mechanisms among gender stakeholders, but they are heavily based on contacts and driven by coordination between individuals, and gaps can occur. Stakeholders noted that ideally MoWA would play a role in facilitating effective EVAW sector coordination, but at present they have not taken an active role in this regard. Stakeholders and partners noted that DFAT is active and engaged in the various platforms for coordinating activities, and the need for a coordination mechanism specifically for donors specifically on VAW was noted, rather than this being a component of wider gender coordination activities. Furthermore, such coordination would enable donors to ensure that programming is covering key gaps, such as those noted regarding service provision through WPCs and WPC capacity building, and ensuring linkages between programming such as support to health sector GBV responses and other stakeholders such as the police, justice sector, and women’s protection services. At the implementation level and based on interviews with provincial entities, there are harmonized EVAW activities between DoWA, MoI, AIHRC and provincial CSOs that run shelters. Similarly, though subnational stakeholders noted receiving training and support through other programming, limited overlap with DFAT EVAW program activities was noted. There were issues in covering more remote districts that requires the attention of partners to harmonize interventions in those areas and maximize collective reach of programming across the entire EVAW sector.
|