Report 4: Interim Evaluation

B.1Overview of the approach

For each sub-theme we have then sought to analyse the evidence available by 'type', with three broad types of evidence used:

Output and expenditure data;

Evaluation and other research evidence;

Primary research.

In undertaking the analysis and synthesis we were mindful of the differences between the different types of evidence and data. Our approach has enabled us to analyse parallel forms of evidence separately and as such has helped to avoid any confusion that may have been created by attempting to synthesise different types of evidence. The amount and type of evidence available differs within themes and sub-themes.

With the evidence analysed separately by type, the next step was to triangulate the evidence to begin to draw some conclusions. This process was done in the following two ways, with the focus on higher level inferences and interactions between the different evidence types:

The first approach was to draw the different sources of evidence together and to comment on what the evidence was inferring around the outcomes for the relevant sub-theme and the additionality of the Games in driving them. In order to robustly comment on outcomes and additionality we drew heavily on the baselines and counterfactuals described in Report 3 of the meta-evaluation.³ In reading, this report (Report 4) a general rule of thumb that can be adopted is that unless stated otherwise the projects, activities and investments reported can be assumed to be additional and delivered as a direct result of the 2012 Games.

The second approach was to draw the different sources of evidence together to begin to provide answers to the various research questions that were established at the outset of the meta-evaluation.⁴ This occurred in a number of ways. For some research questions a number of different evidence sources – within a particular sub-theme – helped in providing an answer, whereas for others an answer was wholly reliant on only one evidence source. There were also some sources of evidence that were particularly useful for helping to answer a number of different research questions, both those within the same sub-theme as well as those outwith. Finally, there were also some research questions for which it was not possible to provide a full answer, either because relevant evidence was not available or because they are focused on a longer timescale and effectively 'out of scope' for this pre-Games interim report.

The final stage involves synthesis of the findings across the themes in order to draw some overall conclusions with regard to the pre-Games legacy of the 2012 Games, through answering the headline and cross-cutting research questions (see Chapter 7).

B.2Gathering the evidence

Output and Expenditure Data: Given potential gaps in the evidence base (see below), the decision was taken that it would be prudent to also collect output and expenditure data for key legacy projects and programmes as this, in the absence of evidence of outcomes and impacts, would at the very least provide insight into both the scale and nature of legacy activities. The process for collecting this data was primarily facilitated through the Evaluation Steering Group whose members include the different organisations and stakeholders largely responsible for the delivery of the 2012 legacy both in London and across the nations and regions. If an organisation was not a member of the steering group it was followed up bi-laterally. In order to ensure consistency of data returns a template was provided by the meta-evaluation team;

Evaluation and Research Evidence: This can broadly be broken down into two types of evidence:

• 
  Evaluation Evidence: Throughout the lifetime of the meta-evaluation, evaluations of 2012 legacy activities have been tracked which has provided us with an understanding of what is likely to be available and when. As such, where evaluation evidence has become available (either interim or final) this has been collected. This has also been supplemented by a similar process of identification, tracking and collection with regard to wider academic research related to the 2012 Games and its legacy;
  
• 
  Primary Research: In a further attempt to reduce gaps in the evidence base a range of primary research was undertaken according to priorities agreed with DCMS. This generally took three forms: it was undertaken directly by the meta-evaluation team; DCMS commissioned specific surveys; or DCMS or the meta-evaluation team worked with an organisation willing to either do the work or to adapt their existing work to assist in filling the gap.
  

The specific sources of evidence used in the analysis are identified within each chapter.