Communication from the Commission to the European Parliament and the Council - Evaluation of the implementation of the IDABC programme - Main contents

Important legal notice

|

Communication from the Commission to the European Parliament and the Council - Evaluation of the implementation of the IDABC programme /* COM/2006/0611 final */

[pic] | COMMISSION OF THE EUROPEAN COMMUNITIES |

Brussels, 24.10.2006

COM(2006) 611 final

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Evaluation of the implementation of the IDABC programme

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Evaluation of the implementation of the IDABC programme

• 1. 

  INTRODUCTION

By Decision of the European Parliament and of the Council (the IDABC Decision)[1] the five year programme on interoperable delivery of pan-European e-Government services to public administrations, businesses and citizens (the IDABC programme) was launched on 1 January 2005 as follow-on to the IDA and IDA II programmes.

The purpose of this communication is to present the findings and recommendations of the evaluation of the implementation of the IDABC programme. The need to carry out the evaluation stems from Article 13(2)-(4) of the IDABC Decision, which also requires the European Commission to forward the results of the evaluation to the European Parliament and the Council at a very early stage in the programme’s lifecycle in order to ensure that the Commission as part of the evaluation would report on the consistency of the amounts for 2007-2009 with the financial perspectives.

The delayed agreement on the financial perspectives 2007-2013 did not allow the Commission to present the results of the evaluation as early as foreseen in the IDABC Decision. However, the Commission can now present the results and confirm that the financial framework foreseen in the IDABC Decision for 2007-2009 is consistent with the financial perspectives and included in the financial programming.

The evaluation was performed by the Commission with the assistance of a team of independent experts from a consultancy company. Representatives of Commission services and Member States have provided their contributions to the methodology and conclusions through a Commission steering group and an advisory group of national experts, as well as through the IDABC management committee (PEGSCO).

The full evaluation report of the consultancy company is available online at http://ec.europa.eu/idabc/en/document/5707/3 .

The evaluation was largely positive describing the IDABC programme as having a global very high quality. However, it highlights a few shortcomings that it recommends for consideration in the further implementation of the programme.

• 2. 

  BACKGROUND

The objective of the IDABC programme is to identify, support and promote the development and establishment of pan-European eGovernment services and the underlying interoperable telematic networks. It is designed to help to achieve targets set in the area of eGovernment by

• - 

  continuing to promote the introduction of information technologies to policy domains, especially where this is facilitated by legislation,

• - 

  building a common infrastructure for cross-border information exchanges between public administrations in order to ensure efficient communications,

• - 

  encouraging the emergence of novel services for businesses and citizens.

For this purpose, the IDABC programme provides funding to actions under two headings, namely Projects of Common Interest and Horizontal Measures, which are listed in the rolling IDABC work programme, to which the Commission at least once a year adopts amendments after consulting the PEGSCO.

Projects of Common Interest are actions in the policy areas of the European Union concerning the establishment or enhancement of pan-European eGovernment services in support of public administrations, businesses and citizens. Such actions, that address European policy requirements and improve cooperation between administrations across Europe, are implemented by the sectors responsible for the different policy areas of the EU, i.e. the Directorates General, and generally monitored by sectoral committees. Projects of Common Interest launched under the programme are described in detail in the IDABC Work Programme[2].

Horizontal Measures are actions designed to support Projects of Common Interest and eGovernment in general. Firstly, they provide and maintain infrastructure services for public services in the Community. Such services provide and maintain technology and software solutions and comprise network services, middleware services, security services and guidelines. Secondly, they initiate, enable and manage the provision of horizontal pan-European eGovernment services to businesses and citizens in Europe, including related organisational and coordination aspects. Horizontal measures launched under the programme are described in detail in the IDABC Work Programme[3]. Horizontal Measures are mostly implemented by the IDABC unit.

• 3. 

  METHODOLOGY

The evaluation relied on multiple methods of investigation, namely desk research, an online survey, interviews as well as case studies, involving a broad range of stakeholders, in all close to 300 representatives of sectoral committees, the PEGSCO and EU services.

• 4. 

  EVALUATION ISSUES AND QUESTIONS

The evaluation focused on the following five main issues:

• - 

  Relevance : To what extent are the IDABC programme's objectives and actions pertinent in relation to the evolving needs and priorities at both national and EU level, first and foremost the needs and priorities crystallizing in the ongoing discussions relating to i2010 and more generally to those established by the Lisbon objectives?

• - 

  Efficiency : How economically are the various inputs and actions being converted into outputs and results? What aspects of the programme are the most efficient or inefficient, especially in terms of resources mobilised?

• - 

  Effectiveness : How far are the IDABC programme’s results and impacts in the process of achieving their specific and general objectives? Are there any aspects that are more or less effective than others, and, if so what lessons can be drawn from this?

• - 

  Utility : How do the IDABC programme’s actions results and impacts, achieved and anticipated, compare with the needs of the target populations? To what extent could measures be taken to improve the utility of the programme and what should these measures be?

• - 

  Coherence : To what extent do the actions form part of a “holistic” approach within the framework of the programme and how well are synergies achieved between programme actions and other Community activities in the area of pan-European eGovernment and infrastructure services?

Apart from specific questions linked to these evaluation issues, three cross-cutting questions were raised in relation to

• - 

  the current state of progress of the actions funded by the IDABC programme, and

• - 

  the co-ordination and involvement of Member States.

• - 

  the extent to which the recommendations of the final evaluation of the IDA II programme have been met in the implementation of the IDABC programme.

• 5. 

  FINDINGS

5.1. Relevance

All the data gathered for the evaluation point to a very satisfactory level of relevance for the IDABC programme. The evaluation found that the IDABC programme’s objectives and planned actions are fully in line with the needs and priorities established at EU level by the Lisbon objectives. The IDABC programme generally contributes directly to broader objectives such as the facilitation of innovation and the uptake of ICT as well as to more specific objectives such as the creation of a more mobile workforce, e.g. by providing information services facilitating movements across borders. However, IDABC also contributes indirectly by increasing the efficiency in exchange of data between Member States and cutting red tape for businesses and citizens. It concludes that the relevance is clearly one of the chief concerns of the IDABC stakeholders and is actively sought in a number of complementary ways at different levels.

5.2. Efficiency

The evaluation notes that an in-depth analysis of efficiency at this stage is premature, given the general and understandably low level of progress of the programme at this early stage. No actual results or impacts are yet visible and measurable. Nonetheless, based on the data gathered from the survey, the evaluation tentatively judges the way the resources are used to implement the activities satisfactory from an economic point of view.

5.3. Effectiveness

The IDABC programme is still in its early stages, and therefore a detailed appraisal of the benefits yielded by the IDABC programme to the Community is not yet feasible. Nonetheless, it is possible to make hypotheses regarding the future evolution of the actions. According to the evaluation, the majority of Projects of Common Interest are currently considered likely to deliver according to schedule, whereas the Horizontal Measures are generally expected to require either extra help or extended durations, primarily due to a late start, lengthy procedures and staff shortage. However, as shown by anecdotal evidence stemming from the case studies, promising levels of effectiveness are likely to be achieved by both Projects of Common Interest and Horizontal Measures.

5.4. Utility

According to the views expressed during the interviews the sort of eGovernment services provided by the IDABC programme are considered unique, as the need for common pan-European telematic interfaces across public administrations is not satisfied by any other programme or initiative in the same manner. Member States that have joined the EU in the last enlargement are in particular extremely satisfied with the services provided, not least since IDABC offers them a way to improve their own telematic infrastructures and services. Put differently, the evaluation concludes that the IDABC programme activities, underlying approach and procedures demonstrate the intentional efforts made towards the satisfaction of the Member States expectations and the priorities set at the European level.

The respondents to the survey view European public administrations, at all levels, as the main target beneficiaries of IDABC activities, with enterprises as the second main target beneficiaries and citizens as the third. However, on the whole, businesses and citizens are at present not as often directly addressed as administrations. Stakeholders do not seem to have a consensual view on whether administrations are a sufficient and satisfactory target group or whether efforts to satisfy needs of businesses and citizens should be increased.

5.5. Coherence

The desk research, the interviews and the views expressed through the survey converge to point at a good level of internal coherence between the measures supported by the IDABC programme. IDABC actions contribute to the programme’s objectives following a holistic approach.

The coherence of the programme vis-à-vis the other Community activities in the area of Pan-European eGovernment and infrastructure services, i.e. the external coherence, deserves more attention, the evaluation observes. Even though the various stakeholders are increasingly making efforts to develop complementarities, when similar types of issues are addressed, the extent to which account is taken of other programmes is sometimes still imprecise, in particular at the level of specific actions.

5.6. The current state of progress of the actions funded by the IDABC programme

On the whole, about 50% of the actions included in the work programme had actually started under the IDABC programme in the first quarter of year 2006. The proportion was slightly higher for the Projects of Common Interest than for the Horizontal Measures. In addition, actions that had actually started were still in an early stage of development. However, the evaluation concludes that the intended IDABC Pan-European eGovernment services, which had either started on time or were carried over from the IDA II programme, are well developed, implemented and used.

5.7. Co-ordination and involvement of Member States

Based upon the views expressed in the interviews and the survey, the establishment, within the Member States, of a national co-ordinated and consolidated approach to Projects of Common Interest appeared to be quite controversial for certain categories of stakeholders. This controversy leads actually to a global level of dissatisfaction as regards the ability to co-ordinate opinions between Member States' representatives in the PEGSCO and in the sectoral committees. However, the evaluation notes that some Member States have implemented specific and efficient means by which the PEGSCO member and the sectoral committee members co-ordinate their views.

Based upon the answers given by the sectoral committee members, the evaluation also concludes that the sectoral committee members’ current level of awareness as regards their responsibilities is not consistent.

5.8. The extent to which the recommendations of the end-term evaluation of the IDA II programme are met in the implementation of the IDABC programme

The recommendations of the IDA II programme were not available until after the adoption of the IDABC Decision. However, based primarily on the findings of the evaluation, most of the thirteen recommendations of the IDA II end-term evaluation are either already taken on board or in the process of being tackled. Only three of the recommendations are still fully applicable, one of which is, however, that priority must be given to those activities that most contribute to programme’s objectives and goals, which is likely to be a constant concern to any programme manager. Not implemented are the recommendations suggesting that

• - 

  a digital dashboard should be created aimed at monitoring and follow-up ongoing and completed activities, both in relation to monitoring the achievement of goals, effects and benefits on a current basis as well as to highlighting and comparing monetary effects to costs,

• - 

  a communication programme should be established targeting EU 15 in the same way as the new Member States had been targeted with a view to ensure buy-in to the way of working and key guidelines/architectures.

With the introduction of a rolling work programme in IDABC the need for a dashboard to monitor the implementation of actions has been reduced considerably as actions are regularly measured against the work programme entries, which are both reflecting the status of the actions and the future planning, and as funds for subsequent phases are only released when milestones have been achieved. At the same time yearly reporting on the implementation of the programme has been introduced and a post-implementation review of each action required within one year of the implementation. Furthermore, to increase the effect of funds allocated, the reuse of components is, to the extent possible, required. The Commission will continue to monitor developments with a view to ensuring a maximisation of benefits.

The Commission will continue to work with Member State representatives in the PEGSCO to ensure buy-in and take-up by focusing on actions covering needs established in co-operation with Member States and sectors.

• 6. 

  CONCLUSIONS AND RECOMMENDATIONS

The evaluation of the implementation of the IDABC programme has lead to a number of conclusions and recommendations derived from these conclusions.

6.1. Timing of the evaluation

It has been difficult for the evaluation, which has the status of an interim evaluation, to establish first effects of the programme as it in reality is still in its very early stages. Comparable problems with timing have occurred in relation to previous evaluations of IDA programmes, which as such makes production or use of evaluation results sub-optimal, the evaluation notes.

Recommendation 1

A greater attention must be paid to the timing of evaluations. Since compliance to the calendar in the IDABC Decision is obligatory, for future programmes a higher degree of flexibility with regard to the timing of the evaluations should be envisaged in the Decision, and consequently enable the programme to benefit from evaluation results more fully.

The Commission is well aware that the present interim evaluation has taken place too early to deliver beneficial results and will for a possible follow-on programme suggest an interim evaluation carried out as mid-term evaluation, as was the case in the Commission’s original proposal for the IDABC programme.

6.2. Organisation of the programme implementation

The difficulties encountered during the implementation of the evaluation, in particular with regard to the description of the overall structure in terms of its stakeholder groups and their relationships, has revealed the inherent complexity of the overall organisation of the programme, especially in the case of Projects of Common Interest. The controversial perception of the co-ordination and involvement of the Member States also highlights the same issue. In concrete terms, the evaluation concludes that sectoral committees and managers need to be fully aware their roles, and the PEGSCO members and members of sectoral committees need to ensure a higher degree of co-operation.

Recommendation 2

The Commission must ensure that all players know their part in the implementation process . The evaluation suggests that organisational diagrams should be designed and operational descriptions of the roles of the contributors adequately disseminated, first and foremost in the case of Projects of Common Interest.

In co-operation with the Member States as represented in the PEGSCO and the sectors as represented in the Inter-Service Group established under the IDABC programme, the Commission will continue to work towards ensuring that all players are aware of their roles and responsibilities in the implementation process.

6.3. Integration of users’ evolving needs into the programme

The evaluation notes that the IDABC programme, like its predecessor, has rightly been adjusted to the evolving European policy priorities. These efforts are appreciated by the Member States that seem to be satisfied with the services provided. However, given the early stage of the programme execution, not all feasibility studies that were planned at the project level have been implemented yet, and therefore fine-tuning the programme to adjust it to the needs of the target populations is still on-going.

Recommendation 3

Efforts should be made at the strategic level of the programme to gather and disseminate specific and up-to-date information about users’ needs. The evaluation suggests that PEGSCO members are solicited as they are an important source of knowledge about needs established at national or international level and, at the same time, that knowledge sharing sessions are organised that enable them to exchange experience and lesson learned with other Member States representatives and with the EU institutions. Data from the project feasibility studies could also be synthesised and disseminated.

In co-operation with Member State representatives in the PEGSCO and sectors, the Commission will continue to refine the focus areas agreed with Member States and confirmed by the “i2010 eGovernment Action Plan: Accelerating eGovernment in Europe for the Benefit of All”[4] with a view to supporting the development of pan-European eGovernment services, also working in co-operation with the eGovernment Sub-Group of the i2010 High Level Group established in the context of the i2010 eGovernment Action Plan.

6.4. Horizontal Measures and human resources

Although the IDABC programme is still in its early stages, certain hypotheses regarding the future evolution of the actions can be extrapolated from the findings of this evaluation. While the majority of Projects of Common Interest are currently considered likely to deliver according to schedule, Horizontal Measures are expected to require either extra help or extended time, primarily due to a late start, lengthy procedures and an unfavourable action/staff ratio.

Recommendation 4

The extent to which IDABC actions are able to comply with agreed milestones should be closely monitored during the implementation of the programme, in particular for Horizontal Measures . In this context the evaluation underlines the need for establishing a more balanced action/staff ratio in the management of the Horizontal Measures, both in relation to the IDABC programme and to any subsequent programme.

In the third revision of the IDABC Work Programme adopted on 14.8.2006[5], the Commission has adapted milestones and deadlines to cater for delays which occurred during the first year of the programme and to take account of the present action/staff ratio. The Commission will continue to monitor the compliance with agreed deadlines and milestones and take action when needed.

6.5. Articulation of the EC eGovernment related programmes

The IDABC programme demonstrates a high level of internal coherence: its actions contribute to the programme’s objectives following a holistic approach ensuring good co-ordination within the programme. On the other hand, more attention needs to be paid to the coherence of the IDABC programme vis-à-vis the other Community activities in the area of Pan-European eGovernment and infrastructure services. Even though the different stakeholders are increasingly making efforts to develop complementarities, when similar types of issues are addressed, the extent to which the other programmes are taken into account is sometimes not clear.

Recommendation 5

A strategic appraisal of the links between the various EU programmes within which the EC develops interoperable eGovernment initiatives should be carried out in order that inter-programme learning processes and synergies can be enhanced. The results of this strategic appraisal should lead to a specification of the strategic positions and inter-relationships of the EC eGovernment related programmes.

In the context of establishing the i2010 eGovernment Action Plan the Commission is ensuring an optimal use of programme resources and the expertise located within the individual programme, creating synergies between the various EU programmes in the area of eGovernment through work-sharing, co-ordination and co-operation. Furthermore, with the expected adoption of the Commission’s proposal for a decision of the European Parliament and of the Council establishing a Competitiveness and Innovation Framework Programme (2007-2013)[6] an ICT Policy Support Programme will be established, gradually integrating several existing programmes in the area of eGovernment (eContent+, eTEN, Modinis). At the same time, the Commission is investigating ways of making established eGovernment solutions programme-independent by defining mechanisms to ensure their financial and operational sustainability as required by the IDABC Decision, likewise with a view to ensuring synergies and increasing cost-efficiency. Through further strengthening of the co-ordination and integration the Commission will continue to ensure the enhancement of inter-programme learning and synergies.

• 7. 

  CONCLUSIONS

While highlighting a few shortcomings that it recommends for consideration in the further implementation of the IDABC programme, the evaluation draws largely positive conclusion, however, noting that a detailed appraisal of the of the IDABC programme is rather premature at the present early stage of implementation.

As outlined above, the Commission will pay the utmost attention to the recommendations of this evaluation.

[1] Decision 2004/387/EC of the European Parliament and of the Council of 21 April 2004 on interoperable delivery of pan-European eGovernment services to public administrations, businesses and citizens (IDABC), OJ L 144, 30.4.2004 (See OJ L 181, 18.5.2004, p. 25. http://eur-lex.europa.eu/LexUriServ/site/en/oj/2004/l_181/l_18120040518en00250035.pdf

[2] C/2006/3606, see http://ec.europa.eu/idabc/en/document/5101

[3] C/2006/3606, see http://ec.europa.eu/idabc/en/document/5101

[4] COM(2006) 173 Final, see http://europa.eu.int/information_society/activities/egovernment_research/doc/highlights/comm_pdf_com_2006_0173_f_en_acte.pdf

[5] C/2006/3606, see http://ec.europa.eu/idabc/en/document/5101

[6] COM(2005) 121 final, see http://ec.europa.eu/enterprise/enterprise_policy/cip/docs/com121_en.pdf

This page is also available in a full version containing de juridische context.