J m barber: What lessons can evaluation of support for Innovation under erdf learn from experience of National Innovation Policies?

• 
  How far the amount spent on grants or subsidies has been kept to a minimum for the results achieved (deadweight kept to a minimum);
  
• 
  How far administration and management (operational) costs have been kept to a minimum for the results achieved.
  

Note that there is a trade-off between these two. Minimising operational input will mean that selection between applications will be rough and ready, that there will a lower proportion of selected projects exhibiting high additionality and excellent value for money, and that the proportion of deadweight will be high. In addition programme managers will have less resources to devote to in-flight monitoring and guidance and there will be a greater risk of supported projects going off the rails. Increasing the amount spent on operating the programme will initially offer marginal high returns but as the amount of operational expenditure increases relative to the size of the programme the marginal return will fall away until it may become negative. One reason for this is that an overly selective and heavily managed programme is likely to deter potential applicants because the application procedures are seen as too onerous and the chances of success too low.

.
68. Economies of scale offer one potential route to efficiency. Having fewer large all-purpose innovation support schemes can lower administration costs, simplify marketing and is more easily understood by outside stakeholders including politicians. However given the highly differentiated innovation characteristics of different sectors and sizes of firms they are less likely to provide the right kind of support in the correct manner to any particular firm. They therefore tend to be less appropriate than a larger number of more specialised programmes and achieve less good value for money. Again ex-post evaluation needs to try and determine whether the correct balance has been struck.

• 
  Technology is usually sector specific and often firm specific. The technology as acquired may need significant adaptation before it can be used by the receiving firm. This will be least true where the technology is relatively mature and acquired via well established supply chain. Where the technology acquired is only part of an overall technology solution e.g. a specific piece of technological information, it will need to be integrated with the other components.
  
• 
  Reports, licenses, instruction manuals etc may not specify the technology in full. Also there may be a significant component of tacit knowledge. Again the problem will be much less in the case of relative mature technologies.
  

73. Firms who are experienced at technological innovation and who have strong in-house technological capability will not usually need much help with the first two phases (some new small high-tech firms may need help on adoption of new business processes). They may however need help in identifying suitable novel technologies and where these might be sourced¹⁷.

74. Innovation support schemes may address one, several or even all of the stages set out in this model. However technology and knowledge transfer must be considered as a holistic process. All the stages must be completed before the economic (and social) benefits of technology transfer are obtain and, being a chain, the process of technology transfer is only as effective as its weakest link. When evaluating a technology or knowledge transfer programme which covers only some of the phases of the framework process it is important to check that the other phases will be carried out on way or another.

75. For example if an campaign is mounted to make SME in the more traditional sectors in a region more aware of a new technology whose adoption might improve their performance then those firms whose interest is aroused must have somewhere to turn for advice and help to acquire, absorb and exploit the technology. Assistance from purely commercial sources may not be appropriate because SMEs may want to proceed cautiously without commitment until they are sure about what they want to do while the consultant etc. will not wish to devote significant resources to a project definition phase if there is a high probability that the potential customer will not continue to full implementation. Surveying the target population of firms should tell evaluators whether or not the firms themselves find a technology transfer programme helpful and expect to benefit but only analysis using a framework like the one proposed here can explain why this the case. For example if a programme attempts to transfer advanced technology or the results of scientific research to SMEs in a mature sector who can neither absorb or exploit it in its native form then it is likely to fail unless an intermediary adapts the technology or science into a form which can be absorbed and exploited by the firms. For example in the UK Birmingham University used the mathematics of chaos to develop a new more effective process for testing industrial springs in a form which the spring producers could use. Subsequent evaluation of the UK Carrier Technology Programme of which it was part highlighted it as a prominent success.

76. The success of public support for innovation will depend on the effectiveness of the means by which it is delivered. Regional innovation programmes may be delivered by central or regional government departments, by public agencies, by universities, by consultants etc. One of the purposes of evaluation should be to assess the effectiveness of the delivery mechanism, what lessons have been learnt for the future and far these have been absorbed by the organisation managing the programme. Ability to manage and deliver programmes can only be built up over time and organisations will only invest in the relevant capabilities if there is an expectation that they can expect to be asked to do so for a longish period.

accumulate knowledge and expertise. Not only that but frequent changes in policy reduce the capacity to learn from evaluation and inhibits learning by policy-makers. Policy experiments have their place but frequent chopping and changing confuses firms and other likely participants resulting in a fall-off in demand. It was a feature of UK innovation policy that new programmes were often undersubscribed but in time applications grew and the designated budget seemed likely to be overwhelmed.

• 
  First, there is a continuing increase in the relative importance of intangible or knowledge assets;
  
• 
  Second, the stock of scientific and technological knowledge is expanding at a rapid rate thereby creating many more opportunities for expanding this stock of knowledge even further through activities such as R&D;
  
• 
  Third, developments in Information and Communication Technologies (ICTs) have revolutionised the storage, transmission, processing and analysis of content, information and data;
  
• 
  Fourth, the number of technologies used in the production of a given product or service is increasing and firms need expertise in a greater range of technologies than before.
  

The key thing to remember is that the knowledge based society is not just about scientific knowledge but all forms of knowledge including knowledge about businesses practices, consumer tastes and behaviour, market developments, online and broadcast content etc. Any programme which improves the generation of knowledge of whatever type, its dissemination and exploitation contributes to the establishment of the KBE and should be judged and evaluated as such.

1. 
   Innovation is a complex interactive process which differs considerable across firms & sectors;
   
2. 
   There are many forms of public support for innovation most of which can be used or delivered at the regional level;
   
3. 
   Evaluation consists of ex-ante appraisal, in-flight monitoring & ex-post evaluation. These should be undertaken as part of an integrated process running from the initial conception of the programme or policy though to the final verdict on its performance which should inform next round of the policy process;
   
4. 
   Both the design & ex-post evaluation of a programme (or policy) should be based on a thorough analysis of the innovation processes it is trying to influence & on the functioning of the relevant elements of the innovation system;
   
5. 
   Ex-post evaluation should consider whether a programme was appropriate (addressed a significant feature of innovation performance & fitted the needs of those it was trying to help), whether it met its objectives, whether it offered value for money & whether it was operated efficiently;
   
6. 
   Ex-post evaluation should also consider how a programme was delivered & the impact on the capabilities & knowledge of the delivery organisation(s).
   
7. 
   Evaluation of programmes which support aspects of technology & knowledge transfer should analyse their operation in the context of a model of the overall process.