One Health Consensus Report Annotation Checklist (OH-CRAC) A cross-sector checklist to support harmonized annotation of surveillance data in reports

To facilitate cross-sector integration of surveillance data it is necessary to improve and harmonize the meta-information provided in surveillance data reports. Cross-sector integration of surveillance results in sector-specific reports is frequently difficult as reports with a focus on a single sector often lack aspects of the relevant meta-information necessary to clarify the surveillance context. Such reporting deficiencies reduce the value of surveillance reports to the One Health community. The One Health Consensus Report Annotation Checklist (OH-CRAC


| INTRODUC TI ON
Good communication of surveillance findings to stakeholders and surveillance actors is the basis for risk assessment and decision support.Growing evidence demonstrates the need to improve the quality of data and annotations of research results, as this is crucial for efficient information uptake into risk assessment and management processes (Simera et al., 2010).Inadequate annotation, where insufficient details are provided, compromises the reliability of the surveillance activity and can prevent report readers from correctly interpreting the surveillance results (Moher et al., 2010).Analysing complex health issues often involves the integration of knowledge from different sectors regarding humans, animals and the environment, which is known as the One Health (OH) approach (Bordier et al., 2018).However, surveillance results are mainly reported by individual One Health sectors (such as public health, animal health or food safety), by datatype (e.g.zoonosis, food-borne outbreak, antimicrobial resistance) or even by individual data categories (e.g.antimicrobial resistance in animals).Consequently, such reports are usually targeted toward stakeholders and experts from the corresponding sector, which generates the risk that meta-information relevant for other sectors is omitted.Despite the efforts carried out in the last years to increase the quality of meta-information in reporting, there is still considerable heterogeneity in the reporting of One Health studies (Davis et al., 2017).
Within the ORION (One health surRveillance Initiative on har-mOnization of data collection and interpretatioN, https://onehe althe jp.eu/jip-orion/) project these challenges were addressed by project partners from food, veterinary or public health institutes from seven European countries leading to the development of the One Health Consensus Report Annotation Checklist (OH-CRAC).
The checklist provides a guideline for annotating the data presented in surveillance reports with the appropriate meta-information.Specifically, OH-CRAC aims at addressing two main objectives: (i) provide guidance to researchers and officials from any One Healthrelated sector regarding what meta-information should be collected and provided in future surveillance reports; (ii) provide a mechanism for mapping such meta-information across sectors to facilitate subsequent integrated analyses.

| MATERIAL S AND ME THODS
The OH-CRAC was developed in a four-stage process represented in Figure 1.In the planning stage (i), a systematic review was performed to identify the critical issues in surveillance reports and define the basis for the checklist creation.In the development stage (ii), a schema to structure meta-information on data in surveillance reports was defined, and for each meta-information item of the checklist a description was defined and examples were provided.During the refinement stage (iii), the OH-CRAC was mapped to existing sector-specific annotation frameworks to (a) validate its cross-sector applicability, (b) identify meta-information items that are used by existing sector-specific best practices but missing from the original OH-CRAC design and (c) provide mappings between OH-CRAC and those sector-specific standards.In the final validation stage (iv), the checklist was tested in pilot studies, and the feedback from these experiences was used to refine and revise the checklist in an iterative manner.Each phase is further described in the following sections.
The research described here involved no human or animal experiments; thus, ethics approval was not required.

| Planning stage
In the first stage, we established a working group with experts from eight different ORION partner organizations from five different European countries providing expertise from the animal health, public health and food safety sectors.This OH-CRAC working group conducted regular web meetings throughout the OH-CRAC development process.Based on the OH-CRAC participants' experiences in their sectors and countries, and the analysis of existing, related sectorspecific guidelines and checklists, this core team identified the main issues and limitations in the meta-information provided on data from surveillance reports.They defined the needs and priorities for the envisaged cross-sector checklist, considering both the target audience and the application scenarios.Another objective was to allow the mapping of surveillance meta-information to one consensus schema, irrespective of the sector or administrative level the surveillance was carried out.Such a mapping to a consensus schema eliminated the need for exhaustive cross-mappings of the different meta-information systems used in the different sectors and administrative levels.

| Development stage
For the development of the OH-CRAC we adopted as a reference the Generic Statistical Business Process Model (GSBPM)

Impacts
• OH-CRAC supports complete and accurate annotation of surveillance results, which is critical for performing integrative One Health analyses.
• The cross-sector design of OH-CRAC supports the annotation of all meta-information relevant for both sector-specific reports and those that integrate data from public health, animal health and food safety.
• The provisioning of OH-CRAC completed checklists as annexes to surveillance data reports and datasets supports the readers from different One Health sectors providing the necessary context to better interpret the surveillance outputs and making the reports more usable and comparable.(ht tps://st at s wiki.unece.org/displ ay/GSBPM/ GSBPM +v5) (UNECE, 2019).GSBPM is a flexible and generic framework that is used by organizations such as Eurostat, UNECE and the OECD to both structure their statistical processes and map information from different sectors, e.g. on administrative, economic and statistical data in official statistics.The GSBPM comprises eight high-level business processes (phases) and within each phase, a number of sub-processes are defined.In order to apply GSBPM in a surveillance data annotation context, we had to align the GSBPM structure to the 'One Health Surveillance Pathway (OHS Pathway)' process model (https://oh-surve illan ce-codex.readthedocs.io/en/latest/2-the-colla borat ion-princ iple.html?highl ight=pathw ay#oh-surve illan ce-pathw ay-visua lization) (Filter et al., 2021).This process model was created within the ORION project to facilitate mutual understanding across sectors on highlevel processes carried out in sector-specific surveillance activities.The OHS pathway also illustrates the flow of information during the different steps of a surveillance activity and where actors from different sectors can interact to promote the One Health paradigm.
We found good alignment between GSBPM and the 'OHS Pathway' (Figure 2), illustrating that GSBPM could be used as a framework to annotate the various tasks within surveillance activities from different OH sectors.Specifically, both frameworks aim to describe all steps relevant for the generation of a report, whether it be a report from a statistical office or, an animal or a public health institute.The analysis showed that the different processes and subprocesses defined in the GSBPM framework match or complement the steps defined in the surveillance pathway model, e.g. on the toplevel phases such as 'Specify needs', 'Design', 'Collect', 'Process' and 'Analyse'.In both frameworks, the applicability of certain phases or sub-processes depends on the type of activity performed and both frameworks allow for interdependencies among sub-processes (Figure 2).However, at this development stage it also became evident that to achieve broad adoption of the OH-CRAC in the One Health sectors, the terminology and the description of certain GSBPM concepts had to be adapted to the terminology used within the One Health sectors.For that reason, in the OH-CRAC schema, the GSBPM phases and sub-processes were renamed as 'annotation sections' and 'meta-information items', respectively.Additionally, examples of possible metadata entries were provided for each item to support end-users.

| Refinement stage
In order to support the provisioning and subsequent integration of meta-information from different OH sectors (e.g.public health, animal health and food safety) as well as from different institutions on federal, national and international levels, a thorough analysis of existing guidelines, checklists and models (listed in Table 1) was performed.Each meta-information item from these existing best practice reporting frameworks was aligned to the corresponding sub-heading in OH-CRAC.In this way, OH-CRAC was updated to also contain meta-information items missing from the original design, thereby ensuring that all meta-information relevant for end-users from the three considered OH sectors would be incorporated.

| Technical implementation
The OH-CRAC team determined that in order to foster the adoption of the checklist by the target community and facilitate the checklist completion process, OH-CRAC should be made available as an online resource.Among the available annotation tools, the only one that met all our requirements (ease of use

| Validation stage
The validation stage aimed at gathering feedback from experts on the expected benefits and usability of OH-CRAC.For this, several pilot studies were carried out during the ORION project at the national level in Germany and Sweden and one with surveillance

| General overview of the OH-CRAC annotation template
The OH-CRAC provides an easy-to-adopt schema to structure the meta-information on the data in surveillance reports in a harmonized way.Following mapping to, and subsequent alignment with the GSBPM schema, the OH-CRAC was structured into five main annotation sections:

| Surveillance needs
The information provided in this section describes what triggered the surveillance activity leading to the results described in the report.This can also include findings from previous surveillance activities.It can also be used to document the results of consultation activities with stakeholders/partners to define their needs and describe the final surveillance objectives.

| Surveillance design
This section is primarily concerned with meta-information related to the development activities, design activities and practical work needed to: (1) define the surveillance outputs, concepts, methodologies, collection instruments and operational processes; (2) design the methodological elements needed to reach these surveillance outputs; (3) specify all relevant meta-information to be collected along the process to support the decision-making process later in the surveillance activity.

| Sample/data collection
This annotation section describes the process of collecting surveillance inputs, which can be actual physical samples with associated metadata; or those inputs that can be defined as sets of data.In either case, the methodological description should detail the different collection modes (e.g.acquisition, collection, extraction, transfer), and how they are loaded into the appropriate environment/system for further processing.

| Data processing
This section specifies the processing of input samples/data and their preparation for analysis.In cases where physical samples are collected as part of the surveillance process, this may include the description of sample processing (laboratorial analysis), which then generates input data (analytical results).The data processing is made up of meta-information items that integrate, classify, check, clean and transform input data, so that they can be analysed and disseminated as surveillance outputs.

| Surveillance output analysis
This section describes how surveillance outputs are produced and examined to ensure that the outputs are 'fit for purpose' prior to dissemination to users.This phase also includes the sub-processes and activities that perform statistical analyses to understand the data and the results produced.
These annotation sections, and their corresponding metainformation items, are outlined in Table 2. Overall, we propose 28 items.An extended table is available in the Data S1 that provides a description and some examples for each item to support the users of the checklist.

| Refinement and validation of the OH-CRAC
The OH-CRAC schema is the result of an iterative development process carried out by the collaborative work effort of the OH-CRAC team.Part of the development process was to systematically review and update the checklist schema, descriptions and vocabulary to meet the requirements of all partners involved.Throughout this process, the alignment of the OH-CRAC to existing guidelines, checklist and frameworks was maintained.The refinement process also led to a desired level of checklist completeness that ensured inclusion of all relevant meta-information critical for all target sectors.
The process also enhanced the comprehensibility of the definitions, provided better meta-information examples and helped to identify those meta-information items where additional sub-items had to be added to provide a better-structured annotation of the surveillance output.An example of this positive evolution of OH-CRAC items is the 'sampling plan' item.Here, as a result of the refinement activities, five additional subcategories were added to support the users in describing work carried out in this surveillance process.The mapping of the existing sector-specific frameworks against OH-CRAC is provided as supplementary information (Data S2).
The validity of the OH-CRAC concept was also evaluated through pilot studies where the OH-CRAC annotation template was filled retrospectively for different existing surveillance reports and the results of the filled checklist were discussed with the creators of these reports.An outcome of this validation process was that the following recommendations to support the completion of the OH-CRAC checklist could be developed: • For those reports where the outcomes of different hazards are described, it was recommended to fill in a single checklist for each hazard.Despite the extra workload, in practice much of the metainformation entered in the checklist can be reused for the different hazards outlined within the same report or between reports when using it for reports generated periodically.
• Although none of the items in the checklist is mandatory, in case there is relevant meta-information missing, the users can also provide links to external resources to support the reader's interpretation.
• Filled checklists can be added as appendices to surveillance reports.In this way, they do not require a change in established report structures.
The execution of the pilots also highlighted the potential benefits of adding the completed annotation checklists to data files where usually no comprehensive data annotation is provided.For example, completed OH-CRAC checklists could be provided in an additional sheet of an Excel data file that users can download from online surveillance data portals.Examples of OH-CRAC completed checklists for different sector-specific report types and datasets are provided as supplementary information (Data S3 and S4, respectively).

| OH-CRAC online tool
In order to facilitate the completion of OH-CRAC, the checklist was made available as an online tool under the RIGOR platform (https:// aflex.vrac.iastate.edu/check list/?t=OH-CRAC).Figure 3 shows a screenshot of the tool.A user uploads a surveillance report as a PDF to the system and tags the meta-information in the text by simply selecting the matching text in the PDF window under the corresponding meta-information item.The RIGOR platform generates as an output the completed OH-CRAC PDF file with the text extracted from the provided PDF file for each meta-information item.In this way, the completion of the checklist can be accomplished in a few hours, depending on the degree of user familiarity with the report and the checklist itself.However, this time estimate might significantly increase if multiple data sets in complex reports need to be annotated or if missing meta-information needs to be gathered from other information sources.

| DISCUSS ION
Complete and correct annotation of research and surveillance results is a critical prerequisite for performing integrative OH analysis.The benefits of adopting a cross-sector approach to describe all meta-information on data in surveillance reports are clear but the traditional and siloed style of reporting within each sector is still a significant barrier.Previous expert groups and systematic and quality observed in the reporting of results (Baum et al., 2017;Davis et al., 2017).In the present study, when analysing the reporting style applied by each sector, we observed that the comparison of results provided in reports is currently limited because most of these results are defined in a sector-specific context.This tendency sometimes leads to missing background information in surveillance reports.Also, the description of applied surveillance methods and in some cases a selective reporting of results could prevent readers from correctly interpreting the outcomes of surveillance activities (Simera et al., 2010).As a consequence, sector-specific reports lacking relevant meta-information have limited ability to contribute to OH-driven analyses.
Recently, many initiatives have been developed to improve the quality and transparency in reporting of research studies related to health.For example, the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network (Simera et al., 2010) or the FAIRsharing platform (Sansone et al., 2019), which collect relevant reporting guidelines, checklists, standards and policies in their databases and reflect the growing interest to improve the quality of reporting.Most of the available guidelines and resources are focused on a particular field.Often, they also primarily target improved consistency of publications in scientific journals by providing sets of rules or principles that guide the users toward more accurate and transparent publications.Given the benefits and the simple adoption plan, many journals have implemented reporting guidelines and checklists for the submissions and editorial processes (Logullo et al., 2020).Nevertheless, the reporting of surveillance results is usually carried out by governmental agencies with a more policyoriented approach (Comin et al., 2019).The provisioning of transparent and consistent surveillance context is the key to establishing and maintaining effective and efficient surveillance and response systems (Calba et al., 2015).In this context, the actors participating at the different levels of the surveillance pathway need to report accurate and consistent details on performed activities, policies and used resources accounting also for the information needs of a variety of different stakeholders (WHO, 2006).
The aim of OH-CRAC is to improve the quality and completeness of both sector-specific reports and those that integrate the data from public health, animal health and food safety.consensus meta-information framework that is compliant with the GSBPM standard (https://stats wiki.unece.org/display/GSBPM/ GSBPM +v5.1) (UNECE, 2019).The GSBPM was developed in response to the needs of official statistical institutions that were facing the problem of insufficient data and metadata harmonization.These institutions have long-term experience with cross-sector data analysis and rely heavily on transparent and complete documentation of meta-information on all activities resulting in official statistics.
The GSBPM has proven to be effective in improving the consistency and comparability of the statistical processes within and between organizations.It also supports decisions on systems architectures and the organization of resources (Bergdahl et al., 2011;Novkovska et al., 2012;Reedman et al., 2010).
The OH-CRAC was designed to support the creators of surveillance reports in different stages of the report generation process; to apply to all report types and datasets and to have utility at both the national and international level regardless of the data source.OH-CRAC can be useful during the planning phase of future disease surveillance programmes as a guidance in the selection of what meta-information should be collected and documented along the whole surveillance process.Report creators can also use the OH-CRAC during the writing process of the report since the metainformation generated in all OH surveillance-related processes can be described and structured in a harmonized way following the OH-CRAC schema.The cross-sector design of OH-CRAC covers the provisioning of all relevant meta-information that is critical for the report's specific sector, but also provides the necessary context to make the report better interpretable, usable and comparable for stakeholders, decision-makers and risk assessors from other sectors and, therefore, from an OH perspective.In this way, future surveillance reports with improved cross-sector comparability and completeness can be generated.Moreover, the provisioning of the OH-CRAC completed checklist together with the surveillance report supports the readers in getting a better overview of the outputs described in the report.
Implementing OH-CRAC is a simple intervention that can improve the quality of surveillance reports but, to be successful, its

| CON CLUS IONS
Given the threats to human health, animal health and food safety globally, there is a pressing need to improve the quality of surveillance reports to support integrated work across the different OH sectors.The adoption of OH-CRAC supports the generation of more complete and transparent surveillance reports.The provisioning of completed OH-CRAC checklists as annexes to the regular report would not require significant changes in the established report generation processes.Those OH-CRAC annexes could even be used as additional metadata in data files and in this way support researchers, risk assessors and decision-makers from different OH-sectors.The foundation of OH-CRAC in the international standard GSBPM also guarantees that meta-information from the OH sector can be linked more easily to other public data that already comply with the GSBPM schema, e.g.data provided by statistical offices.Nevertheless, the main remaining challenge will be to achieve sufficient uptake by relevant researchers and reporting officers.

ACK N OWLED G EM ENT
The ORION project is part of the European Joint Programme One

CO N FLI C T O F I NTE R E S T
None to declare v5.1 conceived by the Joint UNECE (United Nations Economic Commission for Europe)/Eurostat (Statistical office of the European Union)/OECD (Organization for Economic Co-operation and Development) Work Session on Statistical Metadata (METIS) OH-CRAC is explicitly focused on capturing and preserving the context of design, implementation and methodology used in surveillance activities related to OH.Unlike other existing guidelines and checklists focused on improving the reporting of sector-specific surveillance activities, OH-CRAC allows the mapping of existing OH surveillance meta-information from different sectors and levels to a predefined F I G U R E 3 AFLEX tag tool screenshot.The relevant information is extracted from a PDF file uploaded by the user as plain text.The user just needs to select a meta-information item from the checklist displayed on the right and select with the mouse the text that should be included under the selected checklist item.The text is then automatically added to the corresponding OH-CRAC meta-information item indicating also the page where this text is provided in the report.Multiple text entries can be entered per each meta-information item.Once the annotation of the meta-information is finished, the user can download the filled OH-CRAC checklist as a PDF file by clicking on the 'Export checklist to PDF' button.Link to the tool: https://aflex.vrac.iastate.edu/check list/?t=OH-CRAC (VRAC, 2021).Surveillance report used for the figure: https://efsa.onlinelibrary.wiley.com/doi/epdf/10.2903/j.efsa.2020.6007(EFSA& ECDC, 2020) adoption must be supported by regulatory bodies and all the actors involved in the creation of surveillance reports.Despite the benefits of reporting checklists, the major challenge to implement such tools often lies in the adherence of stakeholders, official agencies, institutions and sectors to their own reporting protocols, specifically if new methods would imply extra work in the beginning.By also providing OH-CRAC in a web-based software tool, there is already a technical solution to encourage adoption by researchers and reporting officers.
Health EJP, which has received funding from the European Union's Horizon 2020 research and innovation programme under Grant Agreement No 773830.The authors would like to thank Lesley Larkin, Esther M. Sundermann and Jörn Gethmann for their contribution to the development of OH-CRAC and to the manuscript review.

1 F I G U R E 2 Alignment of the ORION OH surveillance pathway with the
GSBPM (Generic Statistical Business Process Model) model.The GSBPM phases shaded in grey were considered as not relevant for the documentation of performed surveillance tasks and, therefore, were excluded from OH-CRAC 18632378, 2022, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/zph.12947by Danish Technical Knowledge, Wiley Online Library on [16/11/2022].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License from the European Food Safety Authority (EFSA) and European Centre for Disease Control (ECDC).The work plan of pilots included visits, interviews and regular online meetings with the experts from these institutions and agencies.In the first phase of the pilots, we reviewed the specific monitoring or surveillance sys- TA B L E 1 Reporting guidelines and frameworks mapped to the OH-CRAC RISKSUR tools (Design framework and evaluation tool) Animal Health Peyre et al. (2019) experts