Annexes to COM(2016)364 - Final evaluation of the multi-annual EU programme on protecting children using the Internet and other communication technologies (Safer Internet) - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
dossier | COM(2016)364 - Final evaluation of the multi-annual EU programme on protecting children using the Internet and other communication ... |
---|---|
document | COM(2016)364 ![]() |
date | June 6, 2016 |
Civil society was included through actions such as the national youth panels, as well as through the funding given for the eNACSO network of children's welfare NGOs. 12 The aim of this network was to develop a concerted approach by sharing experiences, and by developing joint strategies, in order to ensure that the needs of children are better taken into account in national, European and international discussions around online safety.
Knowledge enhancement is crucial in order to develop robust policy approaches, especially in this rapidly changing area, and increased knowledge on young people's use of new media and technology was a central aim for the work in the Programme. The Programme supported the EUKidsOnline network 13 , whose activities have proven instrumental as a source of accurate trend analyses and for getting input on future research and policy priorities; and a broad range of research projects covering a range of topics 14 . In addition to raising the research profile of Safer Internet related issues, the knowledge developed fed into the knowledge base for all activities in the Programme.
The Programme has also funded a number of studies to gather evidence, specifically the SIP-Bench study 15 and a study on Safer Internet policies and indicators in the Member States 16 . The overall aim of the latter was to improve the understanding of current national policies and actions and to design a sustainable tool enabling identification and sharing of good practices.
The European Strategy for a Better Internet for Children (BIK strategy) 17 has been since its launch in 2012 a central reference document for the activities undertaken in the Programme. The BIK strategy sets out a comprehensive approach through four pillars: stimulating quality content online for young people; stepping up awareness and empowerment; creating a safe environment for children online and fighting against child sexual abuse and exploitation. The Programme contributed to actions under each of these through: the promotion of positive content through national and European competitions; developing and sharing awareness resources and promoting best practices; hotlines and training for dealing with CSAM.
3. EVALUATION OBJECTIVES AND METHODOLOGY
The final evaluation examined the implementation of the Programme, with respect to its relevance, efficiency and effectiveness and assessed its broader impacts and sustainability as well as its EU added value and coherence, which were assessed as parts of these criteria. The evaluation is qualitative in nature and includes limited quantitative evidence but additional data is available in the external evaluation report. The Commission will feed the results of this evaluation into further policy and action development.
To consider the relevance of the Programme the evaluation centred its questions around the evolution of the problems addressed by the Programme and how it responded to these changes, the relevance of the actions and their added value to the key target groups.
The efficiency of the Programme was evaluated through questions about overall satisfaction with the implementation, the efficiency of the activities, the adequacy of funding and its leveraging effect.
Effectiveness considered the contribution the Programme made to protecting children online and the broader digital environment, whether the same results or activities might have been achieved by other more effective means or without the Programme, and what improvements might be made in any follow-up actions.
Impact and sustainability looked at evidence of the sustainability of impacts, whether changes would have happened without EU funding and what would have been the alternatives as sources of funding, and the complementarity/overlap with other national or EU Programmes.
The external evaluation employed a number of data collection tools in order to gather evidence to inform this report. The data collection phase included a survey targeting project participants in the Programme as well as a survey open to a wider range of stakeholders representing all relevant stakeholder groups – industry, public authorities, education, and NGOs.
These were complemented by an interview programme of semi-structured interviews with stakeholders ranging from Commission officials and Programme beneficiaries to external actors such as industry, researchers and Member State and law enforcement representatives. Finally, case studies were carried out illustrating different forms of Safer Internet cooperation in four Member States.
In addition an analysis was carried out of the final review reports of projects, focusing on the extent to which objectives were achieved and their impacts.
3.1.Limitations – robustness of findings
The data collection and analysis carried out has a number of intrinsic limitations, whose impact was mitigated to a maximum possible extent:
• Measuring the impacts of the projects was not possible because of insufficient monitoring tools. The assessment of impact and sustainability was therefore mainly based on the perceptions of project participants, which is often not objective.
• The evaluation takes into account the inherent limitations of the findings of surveys: the answers received reflect the views of a sample of relevant stakeholders and not those of the entire population who has a stake in this domain. Secondly, stakeholders' views convey an individual rather than a holistic perspective.
Based on the elements above, this evaluation has been carried out on the basis of the best available data. Whenever reliable quantitative data is lacking, this is indicated as appropriate and counter balanced with qualitative data and considerations.
4. FINAL EVALUATION FINDINGS
The findings of the evaluation show that the Programme has been successful in achieving its main goals. The success of the Programme was particularly based on its pan-European character and coherence, and the close cooperation between the different stakeholders.
The initiatives funded have had a positive impact on the rights of children. Article 24 of the Charter of Fundamental Rights of the European Union, recognising that children are independent and autonomous rights-holders and prescribes that children have the right to protection and care necessary for their well-being, have been taken into account in the promotion of Safer Internet policies.
The findings presented below by each evaluation criterion represent the consolidated results of the work carried out by the Commission and the surveys and the interviews conducted for the external study.
4.1.Relevance
The results of the evaluation confirm that the implementation of the Programme took account of new technologies and new online behaviours such as children going online at an ever younger age on a variety of connected devices, and that consistent revision in strategy were needed due to the fast changing nature of the field. This was made possible through the flexibility offered by annual work programmes and calls for proposals. Stakeholders acknowledged that the Programme has become a significant forerunner internationally, and has provided leadership as well as financial support to Member States.
This evaluation has also confirmed that the Programme's actions were comprehensive and have both involved and targeted a wide spectrum of different stakeholder groups. 'Increasing public awareness' and 'Fighting against illegal content' were regarded as the most relevant objectives, while stakeholder opinions are more split on 'Promoting a safer online environment' and 'Establishing a knowledge base'. Nevertheless, all four actions were viewed as responding to the needs of children engaged in the Internet. There has been an added value for the Programme's target groups: most stakeholders consulted were not aware of activities similar to their project undertaken by other organisations at national level; and there was a strong indication that without EU funding, the majority of projects would not have been carried out, or at least not to the same extent, due to the lack of financial resources specifically allocated for promoting safer use of online technologies by children.
The evaluation results emphasised the importance of education and positive content. Children need to access quality education that includes a digital component to help protect against various risks when confronted with such rapidly changing technologies. The evaluation underlined the benefits of children's participation and the need both for children to be aware of possibilities offered by digital technologies and for parents to be sufficiently engaged in their children’s online activities. To be active and effective participants online and to be empowered in recognising harmful content and navigating their own safety online, children need digital competences, ideally provided through school curricula.
Programme participants and other policy and industry actors valued highly the research conducted and the importance of disseminating this research throughout the rest of the Programme. Specific input to the survey underlines the need to conduct further in-depth international research on children’s online behaviour 18 , and on efficient methods of education.
Finally, Programme beneficiaries and industry wished to continue their collaboration, including through more specific actions for vulnerable children, disabled children and children with difficult situations at home.
4.2.Efficiency
Project participants reported a high level of satisfaction with the management of the Programme carried out by the Directorate-General Communications Networks, Content and Technology (DG CNECT), though there may be some bias due to vested interest in supporting the Programme. Still, some beneficiaries identified areas for improving the process of applying for future EU grants in the field, such as: simplified procedures and reporting requirements; more flexible requirements to accommodate country specificities; longer-term planning of work and funding; and more information on other funding Programmes in related areas.
Stakeholders generally recognised that the Commission ensured an efficient management of the Programme. The average time to contract was 292 days for the 77 projects that were negotiated during the programming period. Actions of the Programme, and especially the focus on network activities, were largely considered as being ‘good value for money’ but there is scope for more consolidation and collaboration at EU level, across services, and between programme beneficiaries. The findings on the adequacy of funding were split evenly between those who found it sufficient and those who did not, and of those who found it insufficient, nearly all had access to funding from other sources. However, these results could be biased depending on whether the beneficiary had access to additional funding or not.
Funded projects often benefitted from synergies with other initiatives. The fact that SICs exist in 27 Member States is perceived by stakeholders as the main added value of the Programme as this allows the sharing of knowledge and good practice. This is confirmed in the mapping of Safer Internet policies, which concludes that in the majority of the EU countries, the SIC is recognised as the key platform for the implementation and coordination of actions and initiatives. 19
The evaluation confirmed that the projects funded under the Programme have contributed to achieving the Programme’s objectives. This has been done by raising public awareness on safer use of online technologies, by enhancing cooperation between stakeholders, by tackling harmful conduct online, and by involving and informing children in and on online safety. Concerns were however raised that the effectiveness of the Programme and its activities in the future may be endangered by a lack of sustained EU funding and reluctance by industry and national governments to step in.
Regarding improvements for the future, activities which could have received more consideration include social work, and, further research and projects designed to improve children's digital skills. Finally, stakeholders saw the need for industry to play a larger role in enhancing the effectiveness of activities launched by the Programme.
4.3.Impact and Sustainability
The findings from the surveys and interviews with project participants and stakeholders, with the caveat that they could be biased, show that the programme has had a medium to high impact, mainly in raising awareness, fighting illegal content, increasing digital skills and by enhancing cooperation and creating synergies.
Awareness raising, combined with providing education, training and related tools and materials to be used by children, was the most cited key The coordination provided through the Insafe network is seen to maximise synergies and impact, and for example, in 2014 the joint efforts of the network reached out to more than 22,5 million people across (and beyond) Europe. 20
The second strong area of impact was achieved in the fight against illegal content, and specifically the work of the hotlines. The establishment of a system of notice and take-down of CSAM and the improved cooperation between hotlines and law enforcement have led to faster removal. In 2011, 60% of the reported content was removed within 1 to 3 days compared to over 90% in 2014. 21 Also, over the programme period, hotlines received an increasing number of reports 22 leading to an increase in sites deleted by the INHOPE network.
The findings further show that the Programme has been influential in relation to other national and international activities. This is evidenced by references made to research and results from the Programme. European Safer Internet activities are often considered good practice internationally, and have been taken up in Latin America, the US and Asia-Pacific. As an illustration, the Safer Internet Day, which stems from the Programme, is an international yearly event to raise awareness of child online safety. It is organised annually in more than 100 countries and recognised across Europe, North America, South America and the Asia-Pacific region.
The majority of stakeholders considered that project activities would not have been carried out without EU funding, and they voiced concerns about obtaining funding from national governments or the industry and many believed that in this regard, continued funding by the Commission is necessary. Project beneficiaries said that the impact of their projects would continue in the medium to longer term with a crucial role for public authorities and industry in funding future activities.
5. EVALUATION CONCLUSIONS AND RECOMMENDATIONS
On the basis of the external evaluation and its parallel work, the Commission arrives at the following conclusions as input for deliberations on future Safer Internet policy.
–The evaluation shows that the Programme has been successful in achieving its main objectives. The Programme was highly relevant in terms of the problems addressed and the stakeholders involved, and responded well to evolving technologies and problems.
–It is demonstrated that EU intervention had been both necessary and effective in the delivery of results. The pan-European dimension and coherence of the activities, the establishment of SICs in all Member States, and the close cooperation between different stakeholders have contributed to the successful implementation consolidation and collaboration at EU level, across services, and between programme beneficiaries
–The programme was recognised to have been delivered efficiently, although certain procedures were highlighted as presenting scope for simplification.
–The programme has had a medium to high impact, mainly in raising awareness due to the focus on networking and creation of synergies, and fighting illegal content; the establishment of a well-functioning system of notice and take-down of CSAM, complemented by voluntary efforts from industry to counter distribution of CSAM through their infrastructure, has significantly reduced the time to removal. 23
–The Commission has exercised a recognised leadership role both with respect to Member States and internationally where European Safer Internet activities are often considered good practice and have been taken up worldwide such as the celebration of Safer Internet Day.
5.1.Continuation of most effective activities
In order to keep up and build on the results achieved, the following types of activities that have to date proven to be the most effective, could be further sustained:
- the work of the SIC awareness centres, which should be continued by further reinforcing awareness-raising activities, including the dissemination of information and tools for children on safer use of online technologies and continuation of the annual Safer Internet Day; to further expand the work of the helplines and their cooperation with national child protection systems should be encouraged; also the work of the hotlines which has been a major achievement of the Programme should be continued.
- to ensure that the SICs reach out to children in disadvantaged or vulnerable situations (such as children with disabilities, children at risk of poverty, children in migration) allowing them to embrace the benefits of digital technologies by their full inclusion.
-continued research at EU and national level on children’s behaviour online and the risks they face to keep the knowledge base up to date and to ensure any initiatives remain relevant and focused on those areas that need the most attention. Given the lack of focused research in this area, there is a need to think about how to encourage researchers to contribute to the knowledge base on children’s use of the Internet and provide the material for the continuing development of actions, and to ensure that the rights-based approach evident in EUKidsOnline is replicated in future research, with a view to further realising the rights of the child in the online environment. In this context, helplines could be a valuable source of information, as they are the first point of contact, consulted directly by children and parents.
-to continue the mapping of Safer Internet policies on a regular basis to get an overview of how the challenges concerning online safety are met and addressed in policies and initiatives across Europe, and to identify emerging patterns and approaches taken by the Member States.
Overall, the coordinating and supervisory role of the Commission in the implementation of the Programme, and in particular through the SICs has been very valuable for Member States and all stakeholders involved. 24 Thus, a continued coordination role for the Commission in future Programmes would be of added value.
5.2.Facilitate the monitoring of impact of projects
Determining and measuring the impacts of projects co-funded by the EU, is currently not possible because of insufficient monitoring tools. For the future, it is therefore advisable that a robust monitoring system is being set up and that regular independent evaluations are carried out. Therefore, measurable implementation and performance indicators need to continue being further developed, allowing for a comparison and better assessment of projects of the same kind and to highlight good practices.
5.3.Continued need for effective communication
Communication should be pursued between all relevant entities and stakeholders in order to share good practices and to ensure that there is no overlap between future initiatives. In this context, the Commission should further enhance existing cooperation across the various relevant Commission services on issues related to internet safety and full range of illegal content and monitor consistency of their work. In addition, the cooperation and exchange of good practices between different types of stakeholders should be further enhanced.
5.4.Encourage industry and Member States’ involvement
The evaluation has demonstrated that the activities funded under the Programme are valuable, but at the moment there is limited financial involvement from the industry. Given existing concerns of project participants over future funding of initiatives, stronger industry involvement should be sought as alternative sources of funding of activities currently funded by the Commission and, to some extent, by national governments. This could be a long-term objective which would enable the Commission to conduct research on how to best improve the (financial) engagement of industry more generally.
Member States’ involvement in future initiatives should also be further encouraged. Thus, while determining the level of resources for future activities, the availability of national and other sources to complement the EU funding should be assessed at an early stage. Moving forward, the need for national co-financing should be clearly and continuously communicated by the Commission so that no Member States solely rely on EU-funded projects in this policy area. Highlighting good practices of Member States by the Commission and communicating this to all other Member States could incentivise them to increase national funding.
(1) Decision No 1351/2008/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 16 December 2008 establishing a multiannual EU programme on protecting children using the Internet and other communication technologies – OJ L 348, 24.12.2008, p. 118
(2)
Published as: Mapping Safer Internet policies in the Member States: The Better Internet for Kids (BIK) Map ISBN: 978-92-79-47784-3
(3) No 1351/2008/ EC
(4) Regulation (EU) No 1316/2013 of the European Parliament and of the Council of 11 December 2013 establishing the Connecting Europe Facility, amending Regulation (EU) No 913/2010 and repealing Regulations (EC) No 680/2007 and (EC) No 67/2010.
(5) Safer Internet Action Plan (1999-2004) with a budget of 38,7 MEUR; 2005-2008 Safer Internet plus Programme (2005-2008) with a budget of 45 MEUR
(6) https://www.betterinternetforkids.eu/
(7) International Association of Internet Hotlines INHOPE http://www.inhope.org/gns/home.aspx
(8) Directive 2011/93/EU of the European Parliament and of the Council of 13 December 2011 on combating the sexual abuse and sexual exploitation of children and child pornography, and replacing Council Framework Decision 2004/68/JHA
(9) http://ec.europa.eu/dgs/home-affairs/what-we-do/policies/organized-crime-and-human-trafficking/global-alliance-against-child-abuse/index_en.htm
(10) Coalition to make the Internet a better place for kids Statement of purpose. Available at : https://ec.europa.eu/digital-agenda/sites/digital-agenda/files/ceo_coalition_statement.pdf
(11)
European Framework for Safer Mobile Use by Younger Teenagers and Children; Social Networking Principles for the EU
(12) http://www.enacso.eu/
(13) http://www.lse.ac.uk/media@lse/research/EUKidsOnline/Home.aspx
(14) EU NET ADB , SPIRTO, Social Web Social Work, the ROBERT project, Net Children Go Mobile.
(15) SIP-Bench III: http://sipbench.eu
(16) Final report published as: Mapping Safer Internet policies in the Member States: The Better Internet for Kids (BIK) Map ISBN: 978-92-79-47784-3
(17)
(18) An example of this is the study carried out by DG SANCO on marketing to children through social media, online games and mobile applications. The study is expected to publish its results in 2016.
(19) http://www.technopolis-group.com/wp-content/uploads/2014/11/Benchmarking-SI-Policies_Final-report.pdf
(20) Insafe – INHOPE, Annual Report 2014, April 2015
(21) Infographics/INHOPE 2014
(22) Insafe – INHOPE, Annual Report 2013, April 2014, p. 6.
(23) http://www.inhope.org/Libraries/Statistics_Infographics_2014/INHOPE_stats_infographics_for_2014.sflb.ashx
(24) http://www.technopolis-group.com/wp-content/uploads/2014/11/Benchmarking-SI-Policies_Final-report.pdf