Communication from the Commission to the European Parliament and the Council - Final evaluation of the implementation of the IDABC programme - Main contents

Important legal notice

|

Communication from the Commission to the European Parliament and the Council - Final evaluation of the implementation of the IDABC programme /* COM/2009/0247 final */

[pic] | COMMISSION OF THE EUROPEAN COMMUNITIES |

Brussels, 29.5.2009

COM(2009) 247 final

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Final evaluation of the implementation of the IDABC programme

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Final evaluation of the implementation of the IDABC programme

• 1. 

  Introduction

By Decision of the European Parliament and of the Council (the IDABC Decision)[1] the five year programme on interoperable delivery of pan-European e-Government services to public administrations, businesses and citizens (the IDABC programme) was launched on 1 January 2005 as follow-on to the IDA and IDA II programmes.

The purpose of this communication is to present the findings and recommendations of the final evaluation of the implementation of the IDABC programme. The need to carry out the evaluation stems from Article 13(1), (3)-(4) of the IDABC Decision, which also requires the European Commission to forward the results of the evaluation to the European Parliament and the Council.

The evaluation was performed by the Commission with the assistance of an external consultancy company[2]. Representatives of Commission services and Member States have provided their contributions to the methodology and conclusions through a Commission steering group and through the IDABC management committee (PEGSCO).

The full evaluation report of the consultancy company is available online at http://ec.europa.eu/idabc/en/document/5707/3 .

The evaluation report was largely positive describing the IDABC programme as being in line with the eGovernment policy priorities of the European Commission, as expressed in the i2010 strategy[3] and the related eGovernment action plan. The programme plays a unique role to foster the integration of Europe through interoperable public administration and is on track when assessing the implementation. However, the report also offers some suggestions for improvement for future programme management to be taken into account during the implementation of the follow-on programme (ISA)[4].

• 2. 

  Background

The objective of the IDABC programme is to identify, support and promote the development and establishment of pan-European eGovernment services and the underlying interoperable telematic networks. It is designed to help to achieve targets set in the area of eGovernment by

• - 

  continuing to promote the introduction of information technologies to policy domains, especially where this is facilitated by legislation,

• - 

  building a common infrastructure for cross-border information exchanges between public administrations in order to ensure efficient communications,

• - 

  encouraging the emergence of novel services for businesses and citizens.

For this purpose, the IDABC programme provides funding to actions under two headings, namely Projects of Common Interest and Horizontal Measures, which are listed in the rolling IDABC work programme[5], to which the Commission at least once a year adopts amendments after consulting the Member States via the IDABC management committee.

Projects of Common Interest (PCIs) are actions in the policy areas of the European Union concerning the establishment or enhancement of pan-European eGovernment services in support of public administrations, businesses and citizens. Such actions, that address European policy requirements and improve cooperation between administrations across Europe, are implemented by the sectors responsible for the different policy areas of the EU, i.e. the Directorates General, and monitored by sectoral committees. Projects of Common Interest launched under the programme are described in detail in the IDABC Work Programme.

Horizontal Measures (HMs) are actions designed to support Projects of Common Interest but also eGovernment in general. Firstly, they provide and maintain infrastructure services for public services in the Community. Such services provide and maintain technology and software solutions and comprise network services, security services and guidelines. Secondly, they initiate, enable and manage the provision of horizontal pan-European eGovernment services to businesses and citizens in Europe, including related organisational and coordination aspects. Horizontal Measures are mostly implemented by the Directorate-General for Informatics in close collaboration with the other Commission services and monitored by the IDABC management committee. Horizontal measures launched under the programme are described in detail in the IDABC Work Programme.

• 3. 

  Methodology

The evaluation relied on multiple methods of investigation, namely desk research, online surveys, interviews, a focus group meeting with the members of the IDABC management committee as well as case studies, involving a broad range of stakeholders, in all close to 300 representatives of sectoral committees, the IDABC management committee and EU services.

• 4. 

  Evaluation issues and questions

The evaluation focused on the following issues:

• - 

  Relevance : To what extent have the IDABC programme's objectives and actions been pertinent in relation to the evolving needs and priorities at both national and EU level, first and foremost the needs and priorities relating to i2010 and, more generally, those established by the Lisbon objectives? To what extent does the programme contribute to the achievement of the overarching policy objectives?

• - 

  Efficiency : How economically are the various inputs and actions being converted into outputs and results? What aspects of the programme are the most efficient or inefficient, especially in terms of resources mobilised?

• - 

  Effectiveness : How far are the IDABC programme's results and impacts in the process of achieving its specific and general objectives? Are there aspects that are more or less effective than others, and – if there are – what lessons can be drawn from this?

• - 

  Utility : How do the IDABC programme's actions, results and impacts, achieved and anticipated, compare with the needs of the target populations?

• - 

  Sustainability : To what extent is the financial and operational sustainability of infrastructure services ensured? To what extent is the sustainability of horizontal Pan European eGovernment Services and Projects of Common Interest ensured?

• - 

  Coherence : to what extent do the actions form part of a "holistic" approach within the framework of the programme and how well are synergies achieved between programme actions and other Community activities in the area of pan-European eGovernment and infrastructure services? If relevant, to what extent are there spillovers onto other economic, social or environmental policy areas?

Apart from specific questions linked to these evaluation issues, two cross-cutting questions were raised in relation to

• - 

  the current state of progress of the actions funded by the IDABC programme, and

• - 

  the co-ordination with and involvement of Member States.

• 5. 

  FINDINGS

5.1. Relevance

The evaluation report concludes that IDABC contributes to the i2010 initiative of modernising the European public sector by promoting the use of state-of-the-art information and communication technologies, developing common solutions and services and by providing a platform for the exchange of good practice between public administrations. Furthermore, the programme has been adjusted to the evolving European policy priorities expressed in the Manchester and Lisbon ministerial conferences[6] by integrating its actions into the eGovernment action plan[7] and by launching several actions in line with the action plan such as eProcurement (including eOrdering and eInvoicing) and eSignatures. IDABC has become a recognised interlocutor for cross-border interoperability. The programme has contributed to nine sectoral policy objectives, with a particular emphasis on Health and Consumer Protection, Statistics, Employment and Social Affairs and Agriculture[8].

5.2. Efficiency

According to the stakeholders consulted, the programme achieved its objectives efficiently, with an increasing level of efficiency over time. It was noted that the programme experienced a significant delay in 2005, due to the lengthy procurement process for some of the actions. These delays have been partly recuperated due also to the flexibility that the rolling work programme brings, particularly for the reallocation of budget amongst actions.

Furthermore, the shift in balance between the financing of PCIs and of HMs, reflecting directly the decreasing need for PCI funding, is highlighted by the evaluation report as an important development. This shift was foreseeable as today most Community policy initiatives imply some sort of use of ICT solutions and therefore should now cater for the related costs[9].

5.3. Effectiveness

The evaluation team concluded that the IDABC-financed actions have overall contributed to the construction of common infrastructure services for cross border information exchange between public administrations and that the IDABC financed studies have demonstrated a high added value. According to several of the stakeholders, the overall effectiveness of the programme could have been further increased if it had established, since its inception, a top-down strategy[10]. However such a strategy would have to be limited to the scope of the IDABC programme as such. The different policy sectors implementing PCIs act under their own fields of competence and strategy.

Furthermore, it was found that the outcome and uptake of the actions are not systematically reported to the different stakeholders.

5.4. Utility

The evaluators found that the programme responds to the needs of the EU public administrations and that no other programme satisfies the need for the development of common interoperability tools for cross border eGovernment services. It has been remarked by IDABC management committee members that the programme can act as a driving force in the eGovernment area even at national level. With regard to business and citizens, the evaluators found that the IDABC responded to their needs in a more indirect way, mainly because the subsidiarity principle basically implies that relevant services are to be delivered by the Member States.

The European Commission has established stakeholder interaction mechanisms, including meetings, workshops, forums and blogs, but the evaluators found that the understanding of the demands and needs of the stakeholders could still be enhanced.

5.5. Sustainability

Whereas the long-term financial and operational sustainability of PCIs is assured in a sectoral context based on specific sectoral legal bases, the sustainability of IDABC horizontal measures can at present only be ensured via a Community programme. Decisions on ensuring sustainability depend on the actual use and usefulness of a measure. This is therefore an essential element to be considered in the implementation of a follow-on programme.

Furthermore, the evaluation reported that a correct distribution of the available resources between the development of new applications and the maintenance of existing applications is an important issue for the Member States, likewise to be considered in the context of a follow-on programme.

5.6. Coherence

The external coherence of the programme vis-à-vis other Community activities in the area of eGovernment is increasingly well ensured. The collaboration between the IDABC programme and the ICT Policy Support programme of the Competitiveness and Innovation Framework Programme (CIP ICT PSP)[11] has been improving since the mid-term evaluation. Nevertheless, many stakeholders are still not sufficiently familiar with the synergies and complementarities between the different programmes and actions.

As regards the internal coherence, the cross-sectoral integration of horizontal measures is judged as adequate, whereas the coherence between PCIs and HMs could be improved. On the other hand, HMs are widely used outside the PCIs in the wider scope of the IDABC programme.

5.7. The current state of progress of the actions funded by the IDABC programme

According to the fifth revision of the work programme, 54 actions, 22 PCIs and 32 HMs, have been supported by IDABC, totalling around 110 million EUR (around 20 million EUR to PCIs and around 90 million EUR to HMs).

The evaluation concludes that the actions have been developed and implemented smoothly and that no major problems occurred so far. Nevertheless, the implementation of some of the actions is found slightly behind schedule. Delays in some horizontal measures can be partly explained by the circumstances referred to under point 5.2.

Most IDABC actions were still ongoing when the evaluation report was finalised, for which reason their use could not be fully assessed. The evaluators found clear evidence that the actions already implemented were widely used and documented that stakeholders were confident that ongoing actions will be used after their implementation.

5.8. Co-ordination and involvement of Member States

The Member States play an important role in driving the programme and in giving suggestions with regards to future orientations. The representation of national public administrations in the IDABC programme's management committee and in the many expert groups makes the programme a forum also for the coordination of national eGovernment policies. However, the strategic alignment between the national and European priorities is not always optimal as cross border interoperability is currently not of high priority in several Member States.

At the operational programme management level, the evaluation finds that Member States have over the recent years made numerous suggestions for future orientations of the programme and that these suggestions have been taken on board to the extent possible, even if the number of actions directly initiated by the Member States is limited. On the other hand, Member States felt that the way information about the rolling work programme and individual actions is shared between the Commission and the Member States could be improved.

• 6. 

  CONCLUSIONS AND RECOMMENDATIONS

Overall the evaluation concludes that the programme is in line with the eGovernment Policy priorities of the European Commission, plays a unique role within the European instruments to foster the integration of Europe through interoperable public administrations and is on track in the implementation of actions.

Efforts to enhance the efficiency and effectiveness of the programme, some ongoing, should concentrate on:

• - 

  The strategic environment in which the programme should operate[12]

• - 

  The stakeholder communication tools[13]

• - 

  The development of monitoring methods[14]

Furthermore, for future programmes the evaluation recommends that the appropriate contractual frameworks are put in place in due time, to avoid the risk of a delayed start of the actions[15].

6.1. The strategic environment in which the programme should operate

The evaluation suggests raising the programme's profile in order to increase its political standing and enhance awareness at national level[16].

In the context of the proposal for the IDABC follow-on programme, ISA[17], the Commission has suggested to Member States to nominate management committee members at the level of the national Chief Information Officer. This is in line with the evaluation that proposes that the IDABC management committee is strengthened by ensuring a higher-level engagement of senior policy and organisational management.

The Commission is in the process of developing, in close co-operation with the Member States, a European Interoperability Strategy (EIS) which will form the basis of the ISA programme. In addition to the EIS, the ISA programme will be based on the European Interoperability Framework (EIF), which is in the process of being finalised. This is along the lines of the evaluation that recommends the Commission to elaborate a strategic framework based on a problem analysis of interoperability and interchange of data across Europe, comprising a stakeholder analysis as well as a vision and a strategic definition of goals and objectives[18].

Priorities established in the EIS will be translated into concrete actions which will form the basis of the rolling ISA work programme (WP) covering the complete duration of the programme (six years). In this WP the allocation of resources to various priorities, including development and operational activities, will be specified and agreed in close co-operation with the Member States[19].

6.2. The stakeholder communication tools

The evaluation highlights the importance of putting in place appropriate communication and consultation mechanisms for interacting with stakeholders, in particular the policy sectors and the Member States, but also the ICT industry. These mechanisms should be used, as an example, for discussing the sustainability of the different actions[20] with Member States.

The Commission will pursue and intensify the communication efforts made by the IDABC programme in the future ISA programme. In function of the priorities and activities of the programme, a communication strategy will be developed, targeting the various stakeholders[21], such as administrations at various levels, industry, standardisation organisation and open source communities. The Commission will involve these stakeholders in the activities of the ISA programme in function of the issues in question, making use of the most appropriate communication and consultation mechanisms. To that end, the Commission will organise meetings, workshops and conferences and continue with awareness-raising activities. The Commission will also make use of on-line platforms, which allow for wide community-building and provide fora for discussions with and between stakeholders. The Commission will continue and, where necessary, reinforce the use of public consultations for ensuring broad and comprehensive involvement of the relevant stakeholder groups[22].

6.3. The development of monitoring methods

In the ISA proposal[23] the Commission has defined indicators for the ISA programme's general and specific objectives and expected results. The indicators will be the baseline for the Commission to measure the performance of the future ISA programme. This is in line with the evaluation recommendation stating that the Commission should create a monitoring tool for the programme, based on the strategic goals and objectives of the actions it finances[24].

For the monitoring of individual actions, the Commission foresees in the future ISA programme that the phases and milestones of an action are aligned with the Commission's internal methodology used for management of ICT projects. This methodology will facilitate the continuous monitoring and measuring of ISA projects and will facilitate the regular updating of stakeholders on the status of ongoing actions, for instance, on the basis of performance indicators identified at the beginning of the actions. Online reporting practice used currently in the IDABC programme will be continued and reinforced by setting up a solution for continuous monitoring. These activities are in line with the evaluation recommendation stating that stakeholders should be regularly updated on the status of ongoing actions[25], including the sustainability aspects[26]. Finally actions will continue to be implemented on the basis of the rules of public procurement, in the context of which the Commission will endeavour to ensure that the necessary contractual frameworks are in place in due time for the launch of the ISA programme[27].

For the evaluation of the future programme and its actions, the Commission will continue to use its established and proven evaluation process and methods.

• 7. 

  CONCLUSIONS

S everal conclusions and recommendations of the evaluation clearly support the directions taken by the Commission for the ISA programme.

This concerns notably the recommendations on the higher-level engagement of the Member States representatives in the management of the programme and on the elaboration of a strategic framework, which are fully integrated in the Commission's proposal for the ISA programme.

Other recommendations, like the definition of a stakeholder management and communication strategy and that of a monitoring system, are considered by the Commission as pre-requisites for a successful implementation of the ISA programme.

The Commission will take all recommendations into consideration in the implementation of the ISA programme, as well as, for the actions still to be implemented in the remaining period of the IDABC programme, to the extent this is possible in the current time-frame.

[1] Decision 2004/387/EC of the European Parliament and of the Council of 21 April 2004 on interoperable delivery of pan-European eGovernment services to public administrations, businesses and citizens (IDABC), OJ L 144, 30.4.2004 (See OJ L 181, 18.5.2004, p. 25. http://eur-lex.europa.eu/LexUriServ/site/en/oj/2004/l_181/l_18120040518en00250035.pdf

[2] Deloitte Consulting

[3] http://ec.europa.eu/information_society/eeurope/i2010/index_en.htm

[4] Commission's proposal for a decision of the European Parliament and of the Council on interoperability solutions for European public administrations (ISA), COM(2008)583.

[5] C/2008/1881, see http://ec.europa.eu/idabc/en/document/5101

[6] http://ec.europa.eu/information_society/activities/egovernment/conferenceshttp://archive.cabinetoffice.gov.uk/egov2005conference/http://www.egov2007.gov.pt/

[7] http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:52006DC0173:EN:NOT

[8] In addition to competition, education and training, enterprise, environment and internal market.

[9] This is one of the main reasons for which the Commission has proposed to focus only on horizontal actions in the follow-on programme.

[10] Work to develop a European Interoperability Strategy was included in the 5th revision of the IDABC WP in spring 2008.

[11] http://ec.europa.eu/information_society/activities/ict_psp/index_en.htm

[12] Recommendations 1, 2, 3

[13] Recommendations 4, 5, 10, 11 and aspects of 1 and 8

[14] Recommendations 6,7,8

[15] Recommendation 9

[16] Recommendation 1

[17] Proposal for a decision of the European Parliament and of the Council on interoperability solutions for European public administrations (ISA) , COM(2008)583

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:52008PC0583:EN:NOT

[18] Recommendation 2

[19] Recommendation 3

[20] Recommendation 8

[21] Recommendation 4

[22] The above takes into account recommendations 1, 5, 10 and 11

[23] Annex to COM(2008)583

[24] Recommendation 6

[25] Recommendation 7

[26] Recommendation 8

[27] Recommendation 9

This page is also available in a full version containing de juridische context.