Legal provisions of COM(2008)334 - Final evaluation of the eTEN Programme - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
| dossier | COM(2008)334 - Final evaluation of the eTEN Programme. |
|---|---|
| document | COM(2008)334  |
| date | June 3, 2008 |
|
52008DC0334
Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions - Final evaluation of the eTEN Programme /* COM/2008/0334 final */
[pic] | COMMISSION OF THE EUROPEAN COMMUNITIES |
Brussels, 3.6.2008
COM(2008) 334 final
COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS
Final evaluation of the eTEN Programme
(presented by the Commission)
COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS
Final evaluation of the eTEN Programme (Text with EEA relevance)
1. INTRODUCTION
This Communication provides a final evaluation of the eTEN Programme covering the period from re-orientation in 2002 until programme end in December 2006 in response to the requirement expressed in Article 19 of the Financial Regulation covering the programme.
2. BACKGROUND
eTEN evolved from the TENs initiative whose objectives were to provide Trans-European networks in the areas of transport, telecommunications and energy. Developing European transport, telecommunications and energy infrastructures are ambitious objectives and are referred to in the Treaty and the Guidelines for growth and jobs.
The objective of eTEN was to contribute to the development of the Information Society in terms of growth, employment, social cohesion and participation for all in the knowledge-based economy. The main themes covered were eGovernment and eAdministration, eHealth, eInclusion and eLearning. A sixth theme paying particular attention to SMEs was added at a later stage.
Until 2005, all TENs shared a common financial legal base (hereinafter referred to as 'The Financial Regulation'). under which a maximum grant of 10% of the Total Investment Cost could be made. Within this limit eTEN funded projects to a level of 50% for Feasibility and Evaluation Studies, and to a level of 10% for projects involving the deployment of services. The financial legal base for eTEN was revised in 2005 to allow increased grants of up to 30% of the Total Investment Cost for deployment projects.
In conformance with requirements of the Treaty, a set of Guidelines was produced covering the objectives, priorities and broad lines of measures envisaged and identifying projects of common interest (hereinafter referred to as 'The Guidelines'). The Guidelines were modified in 2002 to bring the programme more in line with Community policies and especially with i2010. At the same time the name of the programme was changed from TEN Telecom to eTEN.
Article 19 of the TEN's Financial Regulation requires the Commission to submit to the European Parliament and the Council a comprehensive report on the experience gained with the mechanisms under the Financial Regulation for granting Community aid, in particular the mechanisms and provisions laid down in Article 4. This document responds to this request.
eTEN was implemented through cost-shared projects selected on the basis of public calls for proposals. A total of 149 cost shared projects were established following public calls for proposals with a total funding of just over 160 million €. Projects from the 2003 and 2004 programmes have been completed but those from 2005 and 2006 are still ongoing. All projects are expected to complete by the end of 2009.
A midterm evaluation was completed in 2005 and this covered the results of projects from programmes up to and including 2002 as well as initial results from the eTEN programme which started in 2002.
3. CONDUCT OF THE EVALUATION
The evaluation assessed the following issues: relevance of the programme's objectives, priorities and means of implementation, the effectiveness and impact of the programme, its efficiency and cost effectiveness, its utility and sustainability, and causal links from resources used through to activities and presumed impacts (the intervention logic).
This evaluation was carried out in 2007 by a team of professional evaluators from a company, which was contracted for that purpose. They provided a clear assessment through a thorough analysis of the available evidence consisting of monitoring and evaluation material, and programme documentation and outputs. The evaluators further enhanced the quality of the evidence base by undertaking interviews with key stakeholder representatives. The Commission hereby communicates the results of the evaluation in accordance with the Regulation concerning the programme.
4. EVALUATION RESULTS
Annex 1 of this Communication contains the complete set of Findings provided in the evaluator's report.
The Commission welcomes the evaluation report and its findings confirming both the effectiveness of the programme management and the utility of the programme.
It agrees that the potential for synergy between the eTEN programme and the European Cohesion Fund might have been better exploited.
It also recognises some weaknesses identified by the professional evaluators, notably on the time from closure of call to contract award and long project time-scales. It notes that these weaknesses have however had a minor effect on the overall programme implementation: The Commission has already initiated new services which will reduce such delays in the future.
It welcome the positive assessment of the programme design; the efficiency of the programme management; and the programme's potential to deliver sustainable results, generate positive impacts and add real value by mobilising all of the functions necessary to successful ICT deployment and catalysing structural changes. The results provide also evidence that the Policy Support Programme of the Competitiveness and Innovation Programme (CIP) can be expected to build on the results of eTEN.
It considers that implementing such a unique set of funding mechanisms has been a particular challenge for all actors involved, and welcomes that the evaluators have recognised the efforts which resulted in major improvements in the late stages of the programme.
5. CONCLUSIONS OF THE COMMISSION
The Commission considers that this is a very positive report confirming both the effectiveness of the programme management and the utility of the programme. Some weaknesses have been identified but these have had a minor effect on the overall programme implementation.
Immediate impacts are the involvement of stakeholders from New Member States, SMEs and public bodies favouring the further deployment and uptake of project outputs at a pan-European level and the competitive health of markets for these and related services. Based on the results so far the programme may thus reasonably be expected to produce strong overall impacts by stimulating new areas of activity and demonstrating the ways in which ICTs may contribute to effective policy relating to innovation and the Lisbon Agenda and i2010 objectives. The results provide evidence that the follow on programme, the Policy Support Programme of the CIP, can be expected to build effectively on the results of eTEN.
Work will continue into 2009 by both the evaluators and the Commission to determine the impact of projects from later years of the eTEN Programme. The Commission will report on this work in the midterm evaluation of the Policy Support Programme of the CIP.
Annex 1: Key Findings from the Evaluator's Report
RAND Europe has undertaken a final evaluation of the eTEN programme. This programme nominally finished at the end of 2006 although some project activities contracted late in its life continue. Key findings of the evaluation are as follows:-
Overall success of the Programme
eTEN was ultimately a well run and, in its later stages, successful programme, which contributed to the attainment of its over-arching objectives. This finding is, however, qualified in two ways. Success was not achieved without difficulties, particularly in the early years of the programme. Additionally the extent of its success is not quantifiable – in part because its impacts are not yet fully visible and in part because its overarching objectives were expressed in qualitative terms without specific measurable success criteria.
Programme Implementation Mechanisms
Funding mechanisms other than grant aid were not utilised: this is unsurprising given the modest size of the projects, and we do not consider that this significantly affected the programme’s performance. Linkage to the European Regional Development Funds and the European Social Fund was weak, despite the apparent scope for benefits.
The Market Validation (MV) project mechanism was initially over-utilised (in part because of its more favourable financial arrangements for participants); this was remedied in the later stages of the programme. Many MV projects commissioned before the reorientation did not result in any visible subsequent deployment; this situation also improved in the later stages of the programme.
The Initial Deployment (ID) project mechanism came on stream late, but worked well, particularly following the much-delayed implementation of the 30% cap on overall support as a proportion of total cost. Some promising deployments are now in progress and prospect, including some originating from earlier MV projects.
The funding contracts allowed for insufficient flexibility. Project reviews were consequently limited in their ability to either direct or sanction necessary major changes of direction during project implementation. In interviews, some project managers reported an impression that they had no realistic means of modifying this situation even when the problem had been recognised.
Efficiency of Programme Management
From 2003 no major management issues arose and the standards of management processes; fairness, openness and transparency, and efficiency of day-to-day management were excellent.
The operational procedures for preparing Work Programmes and Calls, their accompanying documentation and the receipt and evaluation of proposals improved continuously throughout the programme. Significant initial problems (particularly with proposal evaluation) were overcome and these mechanisms were subsequently satisfactory and, by the end of the programme, became excellent.
Objective formulation and its reflection in work programme creation were sound and appropriate: we found no reason to criticise the evolution of objectives and indeed consider this responsiveness to change in the programme’s external market (and policy) environment a very positive feature.
The process leading to contracting projects (a sequence consisting of selection, credential checking, and negotiation of the technical annexes to contracts) took too long. This had been noted as a problem by the Intermediate Evaluation, and continued thereafter. In addition, project time-scales were long relative to private-sector counterparts and the dynamics of fast-moving sectors. This, together with the delays in contracting projects, frequently resulted in a very long ‘time to market’, which is likely to have worked against successful deployments and reduced their potential impacts.
Effectiveness, Achievements and Impacts
The programme made considerable progress in involving stakeholders from New Member States, SMEs and public bodies. Their participation strongly favours the further deployment and uptake of project outputs at a pan-European level and the competitive health of markets for these and related services.
Activities in the latter stage of the programme promise good impacts through the formation of sector-based value chain communities (especially in the public sector). The emphasis placed in the latter stages of activity on inclusion in projects of the full value chain including ‘sustaining partners’ and ‘sustaining revenue streams’ was fully validated and represents an important lesson for future programmes.
Fulfilment of Programme Objectives
The later stages of the programme we find to have added real value through actions seeding and/or exemplifying structural change and also through placing actual deployment at the centre of policy development. The programme may thus reasonably be expected to produce strong overall impacts by stimulating new areas of activity and demonstrating the ways in which ICTs may contribute to effective policy relating to innovation and the Lisbon Agenda and i2010 objectives.
Sustainability of Activities
Long-term sustainability of activities is difficult to assess at this stage. However, the involvement of the whole value chain (adopted in the later and more productive stages of the programme), and the specific example provided by some of the projects that have proceeded into wider deployment via these value chains, strongly suggests a positive prognosis.
The Trans-European Network (TEN) initiative is based on articles 154, 155 and 156 of the Treaty establishing the European Community, which set the objective of establishing Trans-European networks in the areas of transport, telecommunications and energy.
COM (2007) 0135 Communication from the Commission – Trans-European networks: Towards an integrated approach {SEC(2007) 374}.
Guidelines for growth and jobs (2005-2008) No 9, 10, 11 and 16.
Regulation (EC) No1655/1999 of the European Parliament and of the Council of 19 July 1999, amending Regulation (EC) No 2236/95 laying down general rules for the granting of Community financial aid in the field of trans-European networks (OJ L 282, 24.11.1995, p. 16).
Article 5 of Regulation No 1655/1999.
Article 4 para 1(a) of Regulation No 1655/1999.
Regulation (EC) No 1159/2005 of the European Parliament and of the Council of 6 July 2005
Decision No 1376/2002/EC of the European Parliament and of the Council of 12 July 2002 amending Decision No 1336/97/EC on a series of Guidelines for trans-European telecommunications networks.(OJ 2002/ L200/1 of 30.07.2002)
COM (2005)354 final, 1 Aug 2005.
RAND Europe, contracted on the basis of a restricted call for tenders issued by DG Information Society in the autumn of 2006.