Designing and implementing Open Access, OD and OS policies Eloy Rodrigues, University of Minho José Carvalho, University of Minho Pedro Príncipe, University of Minho Agenda 1. Welcome, introduction to the workshop, introductions from the participants 2. What elements make successful Open Access, Open Data, Open Science policies and How to design them? – presentation and discussion 3. How to monitor Open Access, Open Data, Open Science policies implementation – use cases and discussion 1. Welcome, introduction to the workshop, introductions from the participants Why do funders define OA and OD policies? Why do funders define OA and OD policies? • Better science • Higher visibility and impact from their funding in research and society in general • Improved monitoring and transparency of the results of funding • Maximized return on investment 2. What elements make successful Open Access, Open Data, Open Science policies and How to design them? presentation and discussion What make policies successful? 1. Content 2. Definition and Implementation Process 3. Monitoring How to design successful policies? • Look around and align… • Make requirements as clear and simple as possible • Provide support (information, infrastructures, and services) • Monitor compliance • Offer incentives (carrots and sticks…) Policy effectiveness  ROARMAP rebuild  122 mandatory policies (institutions)  Repository content measured  Regression analysis carried out using policy criteria Policy conditions recorded in ROARMAP: original set of 13 Condition Must deposit Cannot waive deposit Deposit immediately Must make item OA Cannot waive making item OA Link deposit with research assessment / evaluation Must make item OA immediately Permitted embargo period specified (STEM) Permitted embargo period specified (HaSS) Must retain rights to make item OA Cannot waive retention of rights Age of mandatory policy Requirement for open licensing Policy conditions: eliminated those with small numbers Condition Must deposit Cannot waive deposit Deposit immediately Must make item OA Cannot waive making item OA Link deposit with research assessment / evaluation Must make item OA immediately Permitted embargo period specified (STEM) Permitted embargo period specified (HaSS) Must retain rights to make item OA Cannot waive retention of rights Age of mandatory policy Requirement for open licensing Regression analysis: the final list of important policy criteria  Must deposit  Cannot waive deposit  Research evaluation  Must make deposit Open Access  Cannot waive making item Open Access  Where rights are retained, this cannot be waived Policy conditions: regression analysis Condition Correlation with deposit rate Must deposit ✔ Cannot waive deposit ✔ Must make item OA ✔ Cannot waive making item OA ✔ Link deposit with research assessment / evaluation ✔ Cannot waive retention of rights ✔ Correlation with deposit action Policy criterion Positive correlation Significant correlation Must deposit ✔ ✔ Cannot waive deposit ✔ ✔ Link deposit to research evaluation ✔ ✔ Must make deposit Open Access ✔ Cannot waive making item Open Access ✔ Where policy stipulates authors should retain relevant rights, this cannot be waived ✔ Policies with the significant criteria  18 policies  5 funders  13 institutions Policies with the significant criteria  18 policies  5 funders  13 institutions Must deposit Cannot waive deposit Research evaluation Funders  European Commission: Horizon 2020 policy  Austria: FWF (Fonds zur Foederung der Wissenschaftlichen Forschung)  UK: HEFCE (Higher Education Funding Councils)  US: NIH (National Institutes of Health)  South Africa: National Research Foundation of South Africa Multi-beneficiary General Model Grant Agreement 29.2 Open access to scientific publications 29.3 Open access to research data http://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/amga/h2020-amga_en.pdf Grant Agreement: § 29.2 Open access to scientific publications Each beneficiary must ensure open access (free of charge, online access for any user) to all peer-reviewed scientific publications relating to its results. In particular, it must: (a) as soon as possible and at the latest on publication, deposit a machine-readable electronic copy of the published version or final peer-reviewed manuscript accepted for publication in a repository for scientific publications; Moreover, the beneficiary must aim to deposit at the same time the research data needed to validate the results presented in the deposited scientific publications. (b) ensure open access to the deposited publication — via the repository — at the latest: (i) on publication, if an electronic version is available for free via the publisher, or (ii) within six months of publication (twelve months for publications in the social sciences and humanities) in any other case. (c) ensure open access — via the repository — to the bibliographic metadata that identify the deposited publication. The bibliographic metadata must be in a standard format and must include all of the following: - the terms ["European Union (EU)" and "Horizon 2020"]["Euratom" and Euratom research and training programme 2014-2018"]; - the name of the action, acronym and grant number; - the publication date, and length of embargo period if applicable, and - a persistent identifier. https://goo.gl/eKtAY8 https://goo.gl/eKtAY8 The Content of the Policy Template  The proposed policy is mandatory since research shows that this is the most effective kind of policy and is the most likely to induce compliance with the researchers. PASTEUR4OA and other research shows that if the policy is voluntary, only a small percentage of researchers is expected to comply. A mandatory policy carries an obligation for the funder to monitor its uptake and revise, where necessary, also provisioned in the policy. The Content of the Policy Template  The policy requires self-archiving of peer-reviewed publications in repositories: therefore it does not interfere with the researchers’ freedom of choosing his/her publication venue (journal, monograph publishing venue), while at the same time it gives research institutions the power to manage their own output, using infrastructures (repositories). The Content of the Policy Template  Self-archiving of final author or publisher version of the work should take place immediately upon acceptance for publication and certainly no later than the publication of the work; it is during this phase that the researchers are more likely to perform this task since this is the moment when they are dealing with the publication for the final time. A 6-month or 12-month embargo period for Open Access to the full text of the publication can be accommodated if it is a publisher requirement for Science, Technology, Engineering and Mathematics (STEM) and Social Sciences and Humanities (SSH) fields, respectively. Publication metadata should be openly accessible from the outset. The Content of the Policy Template  The policy requires open access to research data that support and validate publications by archiving them in suitable repositories. This is a policy modeled on that of the most significant research funders around the world, such as the European Commission, the NIH, the NSF, all seven of the UK’s Research Councils, the Gates Foundation, etc. Exception to this policy should be made when necessary (e.g. for reasons of security) and explicitly stated in the guidelines document and in the grant agreement document The Content of the Policy Template  The policy encourages, but does not require, publishing in Open Access journals or monograph series. It renders article processing charges (APCs) and book processing charges (BPCs) are eligible project costs, as well as costs for data management. This entails setting aside the relevant funds and specifying, in the grant application guide the amount that can be spent per project. This is a measure to incentivize the transition to an Open Access publishing system (vs. a subscription system). The requirement for Open Access self-archiving still needs to be met. The Content of the Policy Template  The policy requires that all publications in Open Access journals funded by the funder are licensed under Creative Commons licenses, CC-BY. This will allow true Open Access, in other words, legally re-usable publications and research data (the so-called libre- Open Access). It also recommends CC licenses for items not published in Open Access but deposited in repositories (publications, books, data), without determining the precise licensing terms. The Content of the Policy Template  Compliance with the policy is connected to project reporting, future funding requests and performance evaluation (where applicable). Connecting compliance with project reporting, future funding and performance evaluation is critical in achieving high rates of compliance. The Content of the Policy Template  The policy effectively requires the author to retain the rights necessary to make her/his work Open Access under the requested terms and embargo allowance and deters researchers from the standard practice of transferring their copyright to publishers. It is possible for authors to negotiate with publishers, licensing to them only those rights necessary for the publication, and relevant resources should be provided for researchers (e.g. negotiation tools, author’s contract addenda). Authors are requested to seek another publisher if the proposed publisher does not allow them to meet the terms raised by the funder. The recommended Open Access policy at a glance  Aligns with the European Commission’s H2020 policy  Mandatory  Requires immediate deposit of research outputs in a repository at acceptance for publication but respects reasonable embargoes required by publishers  Encourages publishing in Open Access journals and monographs  Specifies that publishing costs are eligible project costs and grant money may be spent on them: specifies limits that can be spent per project or per year and disallows publication in ‘hybrid’ journals  Specifies the licence to be used for Open Access articles  Links compliance with project reporting, future funding requests and performance evaluation Principles and Recommendations for successful Open Data policies: The RECODE ten overarching recommendations: 1) Develop aligned and comprehensive policies for open access to research data. 2) Ensure appropriate funding for open access to research data. 3) Develop policies and initiatives that offer researchers rewards for providing open access to high quality data. 4) Identify key stakeholders and relevant networks and foster collaborative work for a sustainable ecosystem for open access to research data. 5) Plan for the long-term, sustainable curation and preservation of open access data. 6) Develop comprehensive and collaborative technical and infrastructure solutions that afford open access to and long-term preservation of high-quality research data. 7) Develop technical and scientific quality standards for research data. 8) Require the use of harmonized open licensing frameworks. 9) Systematically address legal and ethical issues arising from open access to research data. 10) Support the transition to open research data through curriculum-development and training. RECODE funders specific recommendations: 1. Develop explicit policies for open access to research data with clear roles and responsibilities. Policies should be consistent with national priorities and aligned with the European framework for open access to research data, while also complementing that for open government data. Provisions should be made for the necessary resources that will allow policy implementation. 2. Adopt a comprehensive approach in funding the implementation of open access to and preservation of research data. Policies will bring the expected results if accompanied by appropriate funds. Particular attention for funding the development and long term sustainability of necessary infrastructures; training of researchers, librarians and other technical staff; innovative actions. RECODE funders specific recommendations: 3. Reinforce the significance of the Data Management Plan (DMP) to embed and promote data management as a distinct activity within the research process. Funder and institutional policies that offer researchers rewards for providing open access to high quality data are central in the transition towards open science. Official measures and processes need to be put in place to include the open sharing of research data in funding. 4. Raise awareness and promote open research data in view of leading an open science paradigm The open access ecosystem comprises a diverse group of stakeholders with multiple and often overlapping functions. To be sustainable, collaboration is essential... Funding agencies RDM guidelines and requirements Australian funders address data in terms of: • data management plans and planning, • data dissemination and sharing, • data access and reuse, • long term data storage. Data sharing policies of the Public Health Research Data Forum: 1. Data management plans 2. Time frames 3. Use of public databases and central repositories 4. Ethics and confidentiality requirements 5. Compliance Policies Key principles: (Review of funders’ data sharing policies of the Public Health Research Data Forum) Of the 20 organizations that make up the forum, 11 have dedicated policies on data sharing and management. 1. Data management plans 2. Time frames 3. Use of public databases and central repositories 4. Ethics and confidentiality requirements 5. Compliance Policies Key principles: 1. Data management plans The majority of the policies considered require or recommend a data management plan to be submitted, as either an integral or recommended part of a research proposal… most at a minimum require researchers to describe the expected data outputs of their research, how these will be collected and stored, and how they will be made available to other researchers. 2. Time frames The question of whether specific time limitations are placed on researchers with regard to the sharing of data varies between funders. Some funders, such state that expected timescales for data sharing should be built into data sharing plans. However, others make explicit expectations for timescales. Policies Key principles: 3. Use of public databases and central repositories Several of the policies reviewed make reference to the use of public databases and/or repositories for the depositing or storage of data, in order for them to be freely accessible to other researchers. 4. Ethics and confidentiality requirements Requirements around ethics and confidentiality, and the protection of research data containing identifiable patient information. A majority of funders cite the importance of ethical conduct with regard to data sharing, and of protecting confidentiality. 5. Compliance How funders will track and ensure compliance with their polices. Provision of data sharing measures as part of a data management plan. Concordat on Open Research Data - principles #1 - Open access to research data is an enabler of high quality research, a facilitator of innovation and safeguards good research practice. #2 - There are sound reasons why the openness of research data may need to be restricted but any restrictions must be justified and justifiable. #3 - Open access to research data carries a significant cost, which should be respected by all parties. #4 - The right of the creators of research data to reasonable first use is recognized. #5 - Use of others’ data should always conform to legal, ethical and regulatory frameworks including appropriate acknowledgement. Concordat on Open Research Data - principles #6 - Good data management is fundamental to all stages of the research process and should be established at the outset. #7 - Data curation is vital to make data useful for others and for long- term preservation of data. #8 - Data supporting publications should be accessible by the publication date and should be in a citeable form. #9 - Support for the development of appropriate data skills is recognised as a responsibility for all stakeholders. #10 - Regular reviews of progress towards open research data should be undertaken. http://ec.europa.eu/research/images/infographics/policy/open-data-2016-w920.png http://ec.europa.eu/research/participants/data/ref/h2020/grants_manual/hi/oa_pilot/h2020-hi-oa-data-mgt_en.pdf Open Research Data in H2020 Open Research Data in H2020 Open Research Data in H2020 Share your views and experience Do you have (or plan to have) different requirements? Why? Do you find challenging to define and implement the recommended policies? Policy implementation: main steps  PREPARATION/CONSULTATION PHASE  Researching international policies is essential to assess position and standing of the funder in terms of policies, infrastructures, practices and degree of participation in international fora.  Assessment of infrastructure (repository) for deposit in Open Access. The policy will be strong if, among other things, the infrastructure that will serve it exists either among all institutions or at the national level. A national harvester can provide a ‘shop window’ for the funder’s research, as well as a means of analysing and monitoring the research it funds. Lack of infrastructures should be addressed.  Assessment of costs and financial planning for the preparation and implementation of the policy, including funding for infrastructure (if this is necessary), funding for APCs, research data management, training and awareness-raising (where applicable). Policy implementation: main steps  POLICY DEVELOPMENT PHASE  Development of policy content  Development of supporting infrastructure.  Development of internal supporting and monitoring mechanism  Preparation of information materials and revision of internal documents  Revision of existing grant application forms, grant agreements, reporting forms, etc., according the requirements of the policy  Preparation of policy supporting actions to help with implementation  Guidelines for applicants with clear instructions on what to do to comply and when and eligible costs  Information materials for grantees on how to comply with the requirements  Awareness-raising actions (workshops etc) Policy implementation: main steps  POLICY IMPLEMENTATION PHASE  Policy adopted by the Funder. A clear and explicit policy from the funder should be made publicly available through the funder’s website as a means of demonstrating the commitment to the principle of Open Access.  Monitoring of compliance by the funder through reporting and other means . Measures should be taken (e.g. holding/reducing/ withdrawal of funding) if compliance is not taking place and non- inclusion of missing deposits in evaluation lists.  Assessment of policy after a couple of years by the funder: revision where appropriate Policy implementation: useful steps  Assessment of the policies of the European Commission and comparator organisations internationally  Dialogue and collaborative approach with stakeholders  Establishment of the relevant e-infrastructure (e.g. repository and/or CRIS/research bibliography)  Formulation of the policy  Guidance and training of researchers in compliance  Provision of incentives and rewards for compliance  Clarification of sanctions for non-compliance  Compliance monitoring mechanism(s)  Provision of resources for the long term sustainability of the services needed to support the policy Practical checklist  Research and map relevant comparator policies  Involve stakeholders  Formulate the policy  Include the clause in grant agreements  Assess infrastructural provision and plan developments where necessary  Guidance and support for researchers  Provision for reward for compliance and sanction for non- compliance  Establish mechanism to monitor compliance  Mechanism to evaluate efficacy of the policy  Resourcing and sustainability plan for supporting the policy 3. How to monitor Open Access, Open Data, Open Science policies implementation use cases and discussion Why Monitor Policies? Why Monitor Open Science Policies? Monitoring policy compliance is important because it enables policymakers to: (i) assess which authors are adhering to the policy; (ii) decide whether additional advocacy practices and communication activities are required to raise awareness about the policy and increase compliance rates; (iii) observe whether any additional infrastructure or tools need to be used to collect evidence on compliance; Why Monitor Open Science Policies? (iv) determine whether any sanctions need to be implemented to enforce a systematic compliance with the policy; (v) and start the process of assessing the benefits that OA is bringing to the institution as levels of OA outputs grow. Why Monitor Open Science Policies? Contribution towards: - improving policy information; - advancing collaboration between stakeholders; - promoting the use of evaluation techniques that provide feedback to revise policies; - informing the assessment of the policy impacts (changing researchers publishing attitudes and behaviors); - helping to link policies to specific outcomes Why Monitor Open Science Policies? More importantly, monitoring a policy compliance is key for universities and funders ‘to account for the outcomes of public spending and to demonstrate return on research investment’ How to Monitor? How to Monitor The key feature of monitoring compliance is to measure the proportion of research outputs that have been made (or will be) Open Access and that effectively comply with the OA policy requirements. The compliance can be determined by a specific year or project, or funding stream. How to Monitor To monitor compliance with OA policies the process must be able to identify: - total number of research outputs subject to the policy, - total number of full-text OA research outputs in the repository, - the number of embargoed full-text items that will become OA at a later date, and whether the research outputs subject to the policy comply with its requirements How to Monitor - Monitor compliance with policy by comparing repository content with indexed records in bibliographic reference databases such as ISI WoS and Scopus, ... - Monitor policy in a systematic way - Involve the researchers / project managers How to Monitor Identify all references between a publication and a project that consider the requisites of the AO policy using: - automatic processes - manual searches - data analysis The process 1 – Identification of the Project 2- Get the Information from sources 3 - Consolidate4 - Report 5 - Analise Monitoring Total Funded Publications Identified Repository Funded Publications Funded Publications in other systems Out of Reach from Monitoring When Monitor? During the project duration At reporting periods At the end of the project + inform, remind during all the process Services to Help Monitoring - Institucional / Thematic Repositories - ORCID - https://members.orcid.org/funders - DOI (CrossRef http://search.crossref.org / DataCite - https://search.datacite.org) - FundRef - http://search.crossref.org/funding - REF Compliance Checker - https://ref.sherpa.ac.uk/ - Lantern - https://lantern.cottagelabs.com/ - CHORUS - http://www.chorusaccess.org/ Services to Help Monitoring - Search Portals: OpenAIRE, Google Scholar, CORE, SHARE - JISC Publications Router - https://pubrouter.jisc.ac.uk/ - JISC Monitor - https://monitor.jisc.ac.uk/uk/ - OpenAIRE Broker Service - InCites - https://clarivate.com/products/incites/ - Danish Open Access Indicator (Monitoring Open Access in a National Context) - http://www.forskningsdatabasen.dk/en - Chronos Open Access - http://chronos-oa.com/ Use Cases H2020 & FCT @openaire_eu OpenAIRE Monitoring Services EC FP7 & H2020 and other national funders Pedro Principe UniversityofMinho (OpenAIRE supportandtraining manager) This is where you type in the event pedroprincipe@sdum.uminho.pt FCT The FCT OA Mandate (main points) • Deposit in IR is mandatory • Allows embargo periods • Allows APC payments • Uses Creative Commons licenses • Aligned internationally • Defined on an existing infrastructure (RCAAP) Overview Repository • Deposit Portal RCAAP • Harvest FCT • Reporting / Monitoring OAI-PMH Validator 1º Search for a Project / Scholarship 74 Repository • Deposit Dspace Addon (Project Authority Control addon) 75 http://api.openaire.eu Gitlab DSpace 5.2 - https://gitlab.fccn.pt/dev-rcaap/dspace/tree/DSpace52++ Input-forms.xml - https://gitlab.fccn.pt/dev-rcaap/dspace/blob/DSpace52++/dspace/config/input-forms.xml Repository • Deposit 2º - Embargoed Items If a publication has embargo, must use a embargo end date . Repository • Deposit 2º - Embargo 07/09/20177 Repository • Deposit 3º - Creative Commons Licence For publications with APC, the use of Creative Commons CC-BY license is mandatory. 07/09/2017 Repository • Deposit FCT set on OAI-PMH 07/09/201779 Repository • Deposit Requisites These requisites are additional to DRIVER Guidelines (mandatory on RCAAP Portal) and use OpenAIRE guidelines to associate publications with projects. 07/09/201780 IMPACT ON SEARCH PORTAL 07/09/2017RCAAP - Repositório Cientifico de Acesso Aberto de Portugal RCAAP Search Portal http://www.rcaap.pt 07/09/2017 Impact on RCAAP Search Portal The monitor of FCT OA Policy The legal deposit of Thesis & Dissertations on the RCAAP Network In both cases, the Portal must now manage different types of access types! 07/09/2017RCAAP - Repositório Cientifico de Acesso Aberto de Portugal RCAAP PORTAL 07/09/2017RCAAP - Repositório Cientifico de Acesso Aberto de Portugal 85 Toolbox for Project Pages Using the Portal information to: – Report - List of publications to report to FCT Funder (HTML; CSV;…) – Widget (integrates the publication list on your blog, website) Put’s the information in context! 07/09/2017RCAAP - Repositório Cientifico de Acesso Aberto de Portugal FCT Scientific Report 07/09/20178 ● Static report – list the publications of a project on a specific date (when generated) ● Automatic Report based on the publications deposited on repositories integrated into RCAAP Search Portal Search by Project 07/09/201788 Filter by project Filter the list of results by project, funding stream or funder 07/09/2017RCAAP - Repositório Cientifico de Acesso Aberto de Portugal Resource Profile Each resource profile indicates the compatibility with the funder requirements 07/09/201790 RCAAP Search Portal API Created for a generic approach, but allows specific contexts focus on the monitoring of the FCT AO Mandate and thesis and dissertations deposit. 07/09/2017RCAAP - Repositório Cientifico de Acesso Aberto de Portugal Example of Request to the API API REST with JSON e XML (CERIF-XML) 07/09/2017RCAAP - Repositório Cientifico de Acesso Aberto de Portugal 07/09/201793 Monitoring all funders (based on OpenAIRE guidelines) Export to CSV 8500 FCT funded publications (3700 in June 2016) Monitoring Dashboard REPORTING AND MONITORING 07/09/201794 Reporting The reporting of publications per project is done automatically (just copy/paste a link) In the future, totally automated and integrated on the Funder applications. 07/09/201795 Monitoring A methodology to analyse/compare the compliance with the mandate is being defined (based on the experience of monitoring a institutional mandate). Several data sources available to search for publications not deposited (not compliant). 07/09/201796 Project Identification Identification of funded publications Compliance Analysis External Sources: OpenAIRE Authenticus Crossref Web of Science Scopus … Search PortalFunder Database Research&Education Statistics Gov. Agency and Funder Monitoring Support Activities - Webinars - Documentation on the websites - Text Mining from OpenAIRE to populate IR’s - Infographics / Decision Trees - Helpdesk 07/09/201798 Conclusions With basic updates, and the adoption of guidelines it’s possible to comply with the funder mandate Different interfaces (human & machine) available to reach the information (user interface, oai-pmh, api) Focus of implementation must be on the Repositories first The update of the project/scholarship list is very important 07/09/201799 Conclusions This approach allows the development of other applications (institutional level / funder level / national level) The same API can be used by different systems (the Curriculum System uses the same API to identify the projects/scholarship, also used to monitor the legal deposit of thesis & dissertations) Conclusions !! - Consider only for the purpose of individual and institutional monitoring, analysis and evaluation of research, publications whose metadata and integral texts are deposited in the institutional repository. - Transform the institutional repository into the official and unique source of information on the scientific production of the institution Share your views and experience How do you monitor your policies? What services and tools are needed/missing for policy compliance? Further Resources The PASTEUR4OA project (http://www.pasteur4oa.eu/) provides an extensive range of briefing papers and Open Access policy guidelines. The Toolkit for Research Funders (http://www.pasteur4oa.eu/resources?qt-resources=9#qt- resources) comprises essential resources to develop and support Open Access policies Thank you www.fosteropenscience.eu Eloy Rodrigues – eloy@sdum.uminho.pt José Carvalho – jcarvalho@sdum.uminho.pt Pedro Príncipe – pedroprincipe@sdum.uminho.pt 07/09/2017106