Lawrence Peter Ampofo
Measurement Approaches for Digital Public Diplomacy
Download 1.29 Mb.
|
Measurement Approaches for Digital Public Diplomacy
The ability to accurately evaluate the success of a digital public diplomacy programme is much easier now that software and methodological processes exist that allow practitioners to examine online behaviour. This, coupled with the analysis of external data, such as information and data from a specific initiative, can generate insights into the effectiveness of the programmes in achieving strategic objectives. Although the Spanish Government is yet to attempt this, the thesis will offer such an analysis of the online behaviour of Spanish language Web users in response to the 11 March 2004 terrorist attacks in Madrid. As such, this thesis is of practical relevance to public diplomacy practitioners who are themselves under pressure to measure the impact of their initiatives. The public diplomacy measurement scholar Ali Fisher argued that ongoing measurement and evaluation of public diplomacy is critical because ‘it underpins evidence-based evaluation, prioritisation, policy planning and spending. This is highly challenging particularly amidst changing priorities and threats, but not uniquely so’ (Wilton Park, 2011: 1). As practitioners are under increasing pressure to deliver successful programmes, it is possible to provide evidence-based analysis of the strengths and weaknesses of specific programmes and for the practitioner to act upon this information quickly and effectively. Indeed, measurement techniques have been employed to a number of counter-terrorism programmes with differing measures of success or key performance indicators (KPI). The effectiveness of digital diplomacy programmes can be assessed by implementing network analysis to examine the differing relationships between data in order to arrive at new insights. Ali Fisher commented that network analysis of digital diplomacy initiatives could yield significant insights, thereby improving the efficacy of such programmes. He argued that ‘[t]o understand online actors and conversations, it is important to recognise the links between them. A network approach, focusing not on the “points” (users of forums) but on the connections between them, reveals both the wider context of jihad online, and the “information horizons”…of the vulnerable or radicalised individuals who are of concern. Self-association between individuals or between individuals and different web forums reveals complex networks, in multi-hub, multi-directional systems...This approach, although apparently theoretical, allows analysts [sic] to move past lists and see connections. With further work this may illustrate the dynamics of online conversations and association – for instance, showing when sites reappear, how users migrate between Jihadi sites (and to and from non-jihadi sites), and the growing or weakening influence of particular individuals or messengers’ (Wilton Park, 2011: 4). However, at this juncture there is inherent difficulty in securing online content for analysis and evaluation. Online content is characteristically ephemeral and subject to deletion or alteration by the site administrator. This poses challenges to practitioners and researchers wishing to analyse the content. This problem can be overcome if regular content aggregation and storage is conducted, although this solution is costly and requires considerable data storage space and security facilities. Indeed, the author had to overcome similar challenges when conducting the internet research for this thesis as outlined in Appendix Two. If, however, relevant volumes of online content can be accessed, sentiment analysis is a useful method of analysing the general disposition of the public towards a particular public diplomacy programme. Items of content are analysed by software programs or human analysts who then assign the item a sentiment ranking from strongly negative to strongly positive. While a more detailed description of the sentiment analysis used for the purpose of this thesis is given in the methodology section for the internet research below, such analysis is not without its problems as the analysis of volumes of content is time consuming and prone to error for both machine and human analysts (Kanayama et al. 2004, Read et al., 2007, O’Connor et al., 2010). It is also possible to conduct attribution analysis of online content by aggregating specific content related to a public diplomacy campaign. This can be achieved using specific online content aggregators such as Sysomos, Alterian or Radian6, which gather online content specifically related to a particular programme or campaign. The content derived from such a search would reveal the number of conversations online that referred to the programme or initiative in question. This kind of analysis is also known as citation analysis and has been used extensively in information science research, during which the importance of academic articles was measured by the number of times they were referenced (Thelwall, 2004). Conducting this kind of analysis will present to the researcher insight on the share of voice a particular programme has in relation to others. This technique will be used in Chapters Four to Eight. In addition, traditional research methods are also used to measure the effectiveness of public programmes or campaigns. Interview research has been used to derive insight of the perceptions of key audience groups of particular campaigns or programmes. Finally, real-time data capture and analysis methodologies can be used to analyse the impact of public diplomacy programmes. Real-time data capture and analysis methodologies describe the implementation of a range of techniques and software tools that allow the researcher and practitioner the ability to aggregate and analyse relevant information as it is created in relation to a particular initiative. This helps the project coordinator react to critical information as it arises and quickly formulate appropriate responses. These techniques, which are in their nascent stages, have demonstrated potential in their ability to provide insight and effective evaluation of public diplomacy programmes and complex emergencies (Coyle & Meier, 2010). This will be employed in Chapter Eight, where real-time data is aggregated and analysed following the death of Osama bin Laden. Directory: portal -> files -> 4733003 files -> Impact case study (ref3b) files -> Ethics and Factual Television: a short history files -> P. O. Box 27, 00014 University of Helsinki, Finland tel. 358-9-191 58080 / 22216; fax 358-9-191 58096 / 23008 files -> The impact of a pledge campaign and the promise of publicity: a randomized controlled trial of charitable donations files -> Diversity and Voice: the Political Participation of Young People in the European Union files -> Chapter 15 a voice in Decision Making: young children in Denmark Stig Broström Introduction Download 1.29 Mb. Share with your friends:
|