Republic of Kosovo Government Ministry of Economic Development
Performance Monitoring and Evaluation
Download 2.66 Mb.
|
- Navigate this page:
- 6.1.Monitoring Evaluation: Background
- 6.2.Monitoring Evaluation System
6.Performance Monitoring and EvaluationBased on the collaborative and agile methodology outlined in chapter 2, we are now turning to the last module or phase in the development cycle of the Kosovo IT Strategy, which is monitoring and evaluation. This module includes the elaboration of an M&E system in order to evaluate the effectiveness of the support measures, to provide feedback and to ensure continuous improvement of the Kosovo IT Strategy. 6.1.Monitoring & Evaluation: BackgroundBefore describing the proposed M&E system for the Kosovo IT Strategy, the terminology concerning monitoring and evaluation needs to be described as well as its importance in the context of strategy development and implementation. Since the expressions “monitoring” and “evaluation” are closely related, they are often used interchangeably. However, it is important to clearly define these two concepts and differentiate between them. According to the Swiss Agency for Development Cooperation (DEZA), “monitoring entails the conscious selection of the areas to be observed, and also the systematic, purposeful collection and production of data and information, during the implementation of a project.”36 Evaluation is defined as “the systematic and objective assessment of an on-going or completed project, programme, policy or strategy, its design, implementation and results.37 Hence, one can conclude that while evaluation is concerned with the assessment of a project or strategy according to specific criteria such as relevance, effectiveness, efficiency or impact, monitoring is a management tool providing the information needed to carry out evaluation. Contrary to monitoring, which relates to a process over a certain period of time, evaluation refers to a specific point in time. Concerning the implementation of the Kosovo IT Strategy, monitoring and evaluation is particularly important for the following reasons:
In the framework of a special strategy module workshop, the working group analyzed different methods and tools for monitoring and evaluation, as well as international examples of M&E systems for IT strategies. Based on this analysis, the working group identified the following key success factors and challenges:
The working group took the above mentioned findings into consideration when developing the monitoring and evaluation system for the Kosovo IT Strategy, which will be presented in the following chapter. 6.2.Monitoring & Evaluation SystemThere are many different methods and tools for establishing a monitoring and evaluation system, including impact-oriented M&E and impact chains, Logical Framework (LogFrame) approach, Balanced Scorecard and so-called rapid appraisal methods. The M&E system for the Kosovo IT Strategy is based on the object-oriented approach of the European Commission’s logical framework (LogFrame) and the concept of impact-chains used by the German Agency for International Cooperation (GIZ). In the following section, we will describe the M&E system for the Kosovo IT Strategy, consisting of the three key elements “structure”, “goal system & indicators” and “methods & tools”. The overall structure of the M&E system is composed of four different levels of goals, as well as the corresponding indicators and means of verification:
At the centre of the M&E system and its structure stands the goal system with its indicators. The overall goal of the strategy represents the first level of the M&E system. Because this goal cannot be achieved through a single measure, sub-goals have been set as “interim targets”. These sub-goals are directly related to the different strategic pillars and can thus be verified more quickly and easily. The sub-goals and their respective indicators are logically related to the achievement of the desired overall goal. Thus, the overall goal of the export promotion strategy is to be accomplished as a consequence of the sub-goals having been reached. Level 3 of the M&E system is represented by the user story & results of the individual tasks, while level 4 represents the activities. It deserves mentioning that level 3 and level 4 are particularly important for monitoring and evaluating the implementation of the operational plan and they constitute the logical basis for achieving the sub-goals, as well as the overall goal. Indicators constitute another important element of the M&E system and have been defined for each of the four levels. According to the Swiss Agency for Development and Co-operation (SDC), an indicator can be defined as a “Quantitative or qualitative factor or variable that provides a simple and reliable mean to measure achievement, results, and to reflect processes as well as changes in the context.”38 With reference to the Kosovo IT Strategy, indicators help simplify complex, IT-related issues and reduce them to an observable dimension. The following table shows the overall M&E system of the Kosovo IT Strategy with its different levels, goals and indicators:
In order to illustrate the M&E system of the Kosovo IT Strategy on level 3 (tasks) and level 4 (activities), the chart below has also been included. It shows a special functionality/section which has been introduced to the operational plan and the M&E system in order to ensure continuous improvement of each task and thus of the overall strategy performance. This is an important element of the agile approach of the strategy. Figure : M&E system level 3 and 4 The above chart also shows the user stories which have been embedded into each task, allowing the establishment and evaluation of consistent impact chains within the M&E system. The M&E system has been designed for an implementation phase of three years. By using the defined indicators, the progress of the strategy implementation can be effectively monitored and evaluated. However, it is important to mention that the M&E system needs to be adapted and updated on a regular basis. Furthermore, responsibilities for M&E have to be assigned. While specific members of the task forces (e.g. knowledge managers) and the steering committee will be responsible for monitoring, evaluation will be conducted with support of external partners. The third element of the M&E system of the Kosovo IT Strategy implies methods and tools for gathering and analyzing the information needed for monitoring the implementation of the strategy. These methods and tools include:
A very important tool for monitoring and evaluating the implementation of the Kosovo IT Strategy will be the Kosovo IT Industry Barometer (KITIB), which has been developed by STIKK in close cooperation with GIZ.39 The following chart describes the KITIB and its functionality: Figure : Kosovo IT Industry Barometer (KITIB) Evaluation will take place at least on an annual basis in the form of joint evaluation workshops, including all relevant stakeholders. During these workshops, the results from monitoring will be evaluated and lessons learnt and improvement measures for the Kosovo IT Strategy deducted. Evaluation of strategy implementation will be conducted according to the following evaluation criteria: Figure : Evaluation criteria The proposed M&E system aims at facilitating effective, time-efficient and impact-oriented M&E, which will in turn promote collaborative learning and continuous improvement of the Kosovo IT Strategy. In addition, it seeks to enhance the agility and strategic response capabilities of stakeholders, thereby increasing the international competitiveness of the Kosovo IT industry and promoting digital transformation. Literature and Sources
Appendix 1 
1 EITO 2014/2015. 2 Gartner (2014): Forecast Analysis: IT Outsourcing, Worldwide, 1Q14 Update 3 EITO 2013: http://www.eito.com/WebRoot/Store15/Shops/63182014/MediaGallery/Categories/Reports/EITO_2013_Definitions_and_Methodology.pdf 4 See GIZ (2011): Manual for IT Sector Promotion in Developing and Emerging Countries. 5 The concept of “Systemic Competitiveness” was originally developed by the German Development Institute (DIE). 6 Scrum is a special agile software development framework. For further information on Scrum please refer to: https://www.scrum.org. 7 Based on Scrum. 8 See Prahalad and Hamel (1990), Porter (1990), Grant (2008). 9 GIZ (2011), Toolbox for IT Sector Promotion in Developing and Emerging Countries: 5 ff. 10 STIKK / GIZ (2014): Kosovo IT Industry Barometer (KITIB). 11 GIZ (2011), Toolbox for IT Sector Promotion in Developing and Emerging Countries: 71 ff. 12 INDEXKOSOVA (2013): ICT Market Analysis and Skills Gap Analysis Kosovo. 13 Assessment value on a scale from 1 = poor to 10 = excellent. Assessment has been conducted by the members of the working group for developing the Kosovo IT Strategy. 14 Source: http://www.art-ks.org/repository/docs/Pasqyre%20e%20tregut%20te%20Komunikimeve%20Elektronike%20TM3%20dhe%20TM4_2012.pdf. 15 STIKK (2013): Internet Penetration and Usage in Kosovo. 16 STIKK (2014). 17 IDC (2012). 18 STIKK (2012), Public Procurement for ICT in the Period June 2009 – June 2012. 19 Kosovo Agency for Statistics (2013). 20 STIKK / INDEXKOSOVA (2013): ICT Market Analysis and Skills Gap Analysis Kosovo. 21 STIKK / INDEXKOSOVA (2013): ICT Market Analysis and Skills Gap Analysis Kosovo. 22 STIKK / INDEXKOSOVA (2013): ICT market analysis and skills gap analysis Kosovo. 23 STIKK / GIZ (2014): Kosovo IT Industry Barometer. 24 Kosovo IT Strategy SharePoint: https://troom-x.capgemini.com/sites/ccdach/itstrategykosovo/default.aspx?PageView=Shared. 25 Sources: EITO 2013, Gartner, CIO.de, Ovum, Silicon.de, PAC, IDC, Computerwoche.de, IT Sourcing Europe, BITKOM, Capgemini. 26 Sources: EITO 2013, PAC, Software in Norway, Marketline, Gartner, Pan-European IT Outsourcing Report, IT Sourcing Europe, Capgemini. 27 Sources: EITO 2013, ICT Market Monitor 2013, IDC, PAC, Gartner, Pan-European IT Outsourcing Intelligence Report, KPMG, IT Sourcing Europe. 28 Porter (1980). 29 Porter (1980), p. 42. 30 Chan and Mauborgne (2005). 31 The results of the Kosovo IT Industry Barometer 2014 have also been included in the formulation of the generic strategy. 32 In the academic literature this option is also often referred to as “client followership”. 33 http://ec.europa.eu/programmes/horizon2020/en/what-horizon-2020. 34 Hubbard, Douglas (2009): 46. 35 Grant (2008): 159. 36 Direktion für Entwicklung und Zusammenarbeit (DEZA) (1999): 22. 37 Organization for Economic Co-operation and Development (OECD): http://www.oecd.org/glossary. 38 Swiss Agency for Development and Co-operation SDC: http://www.deza.admin.ch/ressources/resource_en_23569.pdf 39 For further information on the IT Industry Barometer (ITIB) tool please refer to GIZ (2011), Toolbox for IT Sector Promotion in Developing and Emerging Countries: 71 ff. Directory: repository repository -> Problem-solving and Program Design End-of-topic questions 1 repository -> Multi-objective Performance Optimization of a Probabilistic Similarity/Dissimilarity-based Broadcasting Scheme for Mobile Ad hoc Networks in Disaster Response Scenarios repository -> 10th grade English Language Department Taldykorgan Task for Olympiad 11 repository -> Doc 9718 an/957 Handbook on Radio Frequency Spectrum Requirements for Civil Aviation repository -> Cancun convention & visitors bureau trends and Economy Report – September 2009 repository -> Innovation of ict in Developing Countries repository -> Escaping the trap: the simplified application of eu law repository -> International Journal of Health Promotion and Education Swapping the Lycra for the suit: determinants of cycling for transport among leisure cyclists in Ireland repository -> 8th grade English Language Department Taldykorgan Tasks for Olympiad 8 Download 2.66 Mb. Share with your friends:
|