Article Text
PDF
Abstract
Introduction Previous consensus-based long COVID research has focused on establishing research priorities, developing clinical definitions, core outcomes and a list of recommendations of patient-reported outcome measures that can be used to assess and characterise long COVID. Complementing and extending this work, the proposed study will bring together diverse knowledge users to prioritise concepts of care, quality of life and symptoms to inform a national patient registry on long COVID.
Methods and analysis We will conduct a Delphi process involving Canadians with lived experiences and/or professional expertise with long COVID (including clinicians, policymakers, caregivers and community leaders). A pool of long COVID survey questions has been established through an environmental scan; these questions were coded by topic and will be presented via a series of online, anonymous survey questionnaires to a diverse cohort of 100 participants. Over the course of three Delphi rounds, participants will prioritise and recommend topics related to care, quality of life and symptoms. We will use the prioritised topics to develop a list of core questions as a minimum data set to standardise data collection and inform a national patient registry on long COVID in Canada.
Ethics and dissemination This study has been approved by the University of Saskatchewan Behavioural Research Ethics Board (BEH #4296). Findings will be shared at national conferences and will be published in an open-access peer-reviewed journal. In addition, the minimum data set will be shared with key knowledge users as recommendations to inform a national long COVID patient registry.
- Post-Acute COVID-19 Syndrome
- SARS-CoV-2 Infection
- Research Design
- REGISTRIES
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
The environmental scan conducted for research questions has extensive scope and breadth, covering a wide array of focus areas and topics.
Recruitment through the Long COVID Web network will bring relevance to the minimum data set for a range of knowledge users.
Pilot testing and input from our patient partners resulted in a user-centric survey design to reduce the risk of attrition and improve interpretability of findings.
The Delphi process will achieve consensus only for the identified survey topics, and not for individual questions or their associated response options.
The environmental scan of questions is limited to English-language publications.
Introduction
COVID-19 is a disease caused by infection by SARS-CoV-2. Persons with acute COVID-19 present varying and often debilitating symptoms that appear 1–14 days after exposure and resolve after several weeks,1 in some cases they are asymptomatic. Post-COVID condition (PCC; sometimes referred to as Long COVID (LC) or postacute COVID syndrome),2 is defined by the World Health Organisation (WHO) as a condition which, ‘occurs in individuals with a history of probable or confirmed SARS-CoV-2 infection, usually 3 months from the onset of COVID-19 with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis. Common symptoms include fatigue, shortness of breath, cognitive dysfunction but also others and generally have an impact on everyday functioning. Symptoms may be new onset following initial recovery from an acute COVID-19 episode or persist from the initial illness. Symptoms may also fluctuate or relapse over time’.3 Those living with LC may experience several of over 200 documented LC symptoms, making the diagnosis and characterisation of LC challenging and complex.4 There is no treatment for LC but rapid access to clinically reported data on LC (eg, incidence, symptoms, prevalence and risk factors) is essential to diagnose and manage LC, especially in cases when persons with lived experiences (PWLE) present with atypical or uncommon symptoms. The relevance of such research is especially critical as the burden of LC in Canada remains high. As of June 2023, one in nine Canadian adults reported having experienced long-term COVID-19 symptoms and most continue to experience symptoms.5 Therefore, addressing LC has become a strategic priority for many jurisdictions6 with various LC surveys being conducted across Canada to better understand the symptoms and trajectory of this condition.7–11
Currently, there are limited publicly available data on LC as a new and emerging condition but some work has been done to establish clinical definitions. For instance, the above PCC definition from WHO is the result of work from Soriano et al,3 who used a Delphi method to develop a consensus-based clinical definition within an international panel of 256 patients, clinicians, researchers and WHO staff. Taking a prospective approach, Thaweethai et al utilised data from 9764 participants in a longitudinal cohort study to identify 12 common symptoms of greatest interest present to the majority of the LC patient population to characterise the disease.12 While, Munblit et al involved a panel of 1535 participants from 71 countries in a Delphi process to develop a core outcome set for LC in adults which consists of 11 outcomes within three domains with an additional outcome added a priori.13 Taken together, current research has focused on using common symptoms to define the condition. Building on this, Gorst et al 14 used the Delphi approach to recommend measurement instruments for studying the core outcome domains mentioned in Munblit et al 13 and Ejalonibu et al 15 conducted a review of patient-reported outcome measures to compare, contrast and recommend a combination of condition-specific and symptom-specific tools to measure common outcomes relating to LC. In brief, prior research has established valuable clinical definitions, priority outcomes and core outcome measurement sets for the emerging condition known as LC, and further research is underway to develop specific tools to measure the health status of PWLE relating to COVID-19 and LC outcomes.
In contrast, the proposed study will extend beyond clinical concerns to identify the elements of LC research that are the highest priority in a heterogeneous group of Canadians with lived experiences and/or professional expertise of LC using a Delphi process. The resultant data will be used to establish a minimum data set for use as a parsimonious tool for capturing data worth investigating and reporting on cross-nationally in a LC patient registry. A minimum data set in this study is defined as the least number of elements recommended to be included in a LC patient registry based on the current state of knowledge; it can be understood as ‘a coherent set of data elements which should be collected for specific categories or domains of healthcare’.16 By compiling this data for use in a national patient registry within Canada, researchers will be better equipped to observe the course of the disease, understand variations in treatment and outcomes and monitor the appropriateness of care provided.17 Moreover, the registry will align and extend regional (eg, provincial) research activities while still allowing flexibility in each region when setting its own research agenda. This registry will benefit both clinicians and clinical researchers, and also policymakers, public health officials, population health researchers and PWLE across Canada.
Aims and objectives
The goal of this study is to use the Delphi method to develop consensus on strategic topics related to LC care, symptoms and quality of life. Subsequently, these topics will be translated into 50 survey questions as a ‘minimum data set’ to inform the development of a Canadian patient registry on LC.
Methods
Delphi method
The Delphi method is a research technique that encourages knowledge sharing to facilitate decision-making. It was created by the RAND Corporation in the 1950s18 and, at that time, was used to deliberate strategic military operations. Since then, this technique has been applied to various other fields of study19 and is especially useful in situations where there is a level of uncertainty surrounding how something works or how it is defined.20 In the Delphi process, individuals with in-depth knowledge of a topic participate in a series of questionnaires. They provide ratings and feedback on each question presented based on their individual expertise. All ratings are then averaged and analysed, and a decision is made to include or exclude the survey item by way of consensus.21 Past studies show the Delphi to be an effective means of establishing minimum data sets.21
Patient and public involvement strategy
The Long COVID Web (LCW) is an online group of nearly 800 Canadian scientists, clinicians, PWLE, knowledge keepers, policy and decision-makers, industry experts and knowledge users in the field of LC.22 To shape the Delphi process and ensure its success, three groups (LC researchers, clinicians and PWLE) are participating in the development and implementation of this study. The steering committee is made up of members of the LCW and includes individuals with Delphi method expertise; they direct the working group on best research practices, especially in the study design phase. The advising committee has domain knowledge and is also part of the LCW. Its members make decisions for this research based on resource availability and evidence in the LC field. Our target population for this study are PWLE and/or professional experience with LC; this includes clinicians, researchers, healthcare professionals, caregivers, family or friends of patients and patients. The working group includes researchers and PWLE; it carries out research activities and provides communications support to drive the study forward, drawing on guidance from the steering and advising committees. Members of each group will be permitted to participate in the Delphi (presuming they meet study criteria), excluding members of the working group responsible for data analysis. PWLE participated in the pilot testing of the survey questionnaire to improve its ease of use and feasibility for completion.
Designing the Delphi questionnaire
The purpose of the Delphi questionnaire is to identify the level of agreement for including various topics within each focus area in a minimum data set that can be used for LC patient registries nationally in Canada.
Phase one
In the first phase of survey development, the working group conducted an environmental scan between October 2022 and December 2022 using PubMed, MEDLINE, University of Manitoba Libraries and Google Scholar with predefined search criteria to collate a comprehensive repository of LC survey questions being asked worldwide in the English-language. Search terms included:
Any of LC or PCC or postacute COVID-19 syndrome or postacute sequelae COVID.
Any of surveys or patient survey or registry or patient registry or questionnaires.
Two reviewers from the working group completed the original search, identifying and documenting references of interest, in addition to reviewing individual references that were provided externally from the environmental scan.
Reviewer 1 (AAm): 133 articles and two published web-based survey questionnaires were retrieved and screened.
Reviewer 2 (SS): 171 articles were retrieved and screened, along with 14 articles provided through other sources.
Information concerning the country of origin, year of publication, journal of publication, MeSH terms and keywords were documented to enable location of other LC survey questionnaires in secondary searches of the literature.
Survey questionnaires solely pertaining to acute COVID-19 were excluded from the repository. After reviewing each reference for inclusion and removing duplicates from the joint search, 32 references were found to fit our search criteria, such that in the first phase of survey development, we retrieved a total of 1390 survey questions focused on LC. These 1390 questions were recorded along with the type and format of each answer option (ie, free fill, Likert scale, multiple choice and single choice). This work was presented at the first Canadian Symposium on Long COVID.23
Phase two
In the second phase of survey development, screening took place; members of the working group removed similar or duplicate questions from the repository, reducing the total number of questions to 791. For example, if a question about age was asked in the survey, all other questions relating to age were aggregated into a singular question. In addition, the following question types were deemed to be out of scope for this study: perspectives of family/caregivers to PWLE (as this information would be captured by other questions in the survey), pre-existing symptoms unrelated to COVID-19, recommendations for rehabilitation and laboratory testing.
Questions about general COVID-19 experiences, demographic information, consent, contact information and interest in snowball recruitment were removed from the repository and later integrated into the demographic questionnaire to support the selection of participants.
However, given the many questions remaining in the repository after the second phase was complete, the advising committee decided it would be most effective to conduct a Delphi study focused only on care, quality of life and symptoms, resulting in a final total of 635 questions. Care was defined as community supports, health services and treatments; quality of life was defined as physical, social and emotional well-being and ability to perform activities of daily life; symptoms was defined as the indications or signs of a disease.
These questions were reviewed for clarity by members of the working group and the advising committee, and concerns were addressed by the working group (eg, adding definitions of medical terms, fixing grammatical errors, substituting difficult terms with more accessible terms, adding examples of vague categories and adding practice items for participants to familiarise themselves with the platform and question format).
Phase three
In the third phase of survey development, efforts were made to reduce cognitive and time burdens for participants. First, response options for questions were excluded from the survey questionnaire. Second, questions were adapted to include definitions of medical terms and use more accessible language whenever possible. Third, participants will be informed that they can complete this round in multiple sittings, save their progress and receive a reminder to complete their survey questionnaire if it remained partially complete after a week.
However, this still did not reduce the number of questions that participants were expected to rate and given that many people with LC lived experiences suffer from brain fog, fatigue and postexertional malaise, we simplified the survey. Instead, during the first round of the Delphi study, we will ask participants to prioritise the inclusion of topics within each focus area relating to care, quality of life and symptoms instead of listing individual questions. The survey topics were reviewed collectively by researchers within the working group to reduce bias, and ensure that all key concepts from the individual questions were captured. For a list of survey topics and an overview of the survey questionnaire, see online supplemental file 1.
Supplemental material
Recruitment strategy
Recruitment for this study will be facilitated through electronic means via an online post and email message to the LCW. Recruiting from this group will leverage its diversity and high levels of engagement, while simultaneously allowing the effective management of resources (eg, completing follow-up emails and phone calls). Participants will be compensated $100 for completion of the first round, $50 for the second round and $50 for the third round. If this recruitment strategy fails to identify a diverse group of stakeholders and 100 participants cannot be attained, recruitment efforts will be extended through word of mouth and strategic emails to networks, patient-oriented research support groups and academic institutions across Canada.
Study participants
One hundred participants with lived experiences (patients, friends or family of patients and caregivers) and/or professional expertise (researchers, clinicians and other healthcare professionals) with LC will be recruited for this study. Inclusion requirements will be that participants be 18 years of age or older, have internet access, are able to access documents online, and have the capability to read and understand English. Equity, diversity and inclusion principles will be followed to ensure a diverse and representative sampling of the Canadian population. To achieve this, demographic information will be gathered through a questionnaire sent to potential participants (see online supplemental file 2). Variables of interest include age, gender, province of residence, residential community type, ethnicity, employment status, confirmation of COVID-19 diagnosis and nature of involvement with the LCW. Interested participants with documented evidence of COVID-19 diagnosis will be prioritised, and enrolled into the study using stratified sampling techniques. This is to ensure that there is variation within our sample with consideration for under-represented groups.
Supplemental material
Ethics and dissemination
This project was reviewed and approved by the Behavioural Research Ethics Board at the University of Saskatchewan (BEH #4296). Findings will be shared at national conferences and published in a peer-reviewed journal. In addition, a report will be shared with all key knowledge users to recommend the implementation of a national LC patient registry in Canada.
Number of participants
The number of experts included in past Delphi studies has ranged widely, with a median of 17 and a mean of 40.24 Past research has also noted that a response rate of at least 40% is typically expected with a margin of error of 5% or less25 but, as there is no guidance for what constitutes an acceptable response rate,26 one of 50% is anticipated throughout this study. Survey dropout and survey fatigue between rounds may be an obstacle to gaining meaningful data from this study; this is accounted for by targeting 100 participants in the initial phase. Following Hall et al’s27 recommendations for reducing the rate of attrition, participants will be provided with set expectations about time commitments, email reminders and 1 week extensions, if needed, in the effort to maintain a high response rate.
Designing the Delphi process
Utilising ACcurate COnsensus Reporting Document (ACCORD),28 Core Outcomes Set-Standards for Development (COS-STAD)29 and the Core Outcome Measures in Effectiveness Trials (COMET) handbook,30 we determined the purpose of each round of the study and outlined the methodology we will use to achieve our outcomes. Figure 1 summarises the Delphi process to be followed, which will involve a maximum of three rounds to decrease the propensity for participant drop-out.24 27
Summary of the process utilised to create the long COVID minimum data set.
Administration of the Delphi questionnaire
This study is planned to start in May 2024 and end in November 2024. The survey questionnaires will be delivered using SurveyMonkey between May and September 2024. Participants wishing to have a printable version of the survey questionnaires will have the option to receive a PDF version of the questions; however, responses will need to be submitted via SurveyMonkey. Quantitative and qualitative data analysis will be ongoing between survey rounds. The development of the minimum data set and dissemination of the final report will be completed by November 2024.
First round
The purpose of the first round is to eliminate survey topics not essential to the minimum data set. Topics will be eliminated based on participants’ responses to the question, ‘How strongly do you agree or disagree that the following (survey element) should be part of a Long COVID minimum data set?’ within each focus area (care, quality of life and symptoms). Responses will be recorded using a five-point Likert scale rating consisting of the following: (1) Strongly disagree; (2) Disagree; (3) Neither agree nor disagree; (4) Agree and (5) Strongly agree. A review of studies using the Delphi technique found the cut-off threshold for agreement in a number of cases to be 60% or higher.24 Given our high volume of topics (survey elements), we will follow a similar but more stringent guideline; topics receiving 70% or more ‘agree’ and ‘strongly agree’ responses will move to the next round of the study. Statistical measures of central tendency (median) and spread (IQR) will be used assess consensus with a maximum spread of ≤1.25. According to Beiderbeck et al, a threshold of a maximum of 25% variation of the Likert scale (1.25 on a scale of 1–5) can serve as an indicator of consensus.31
Open textboxes will be available on each page providing opportunities to suggest additional survey topics and rewording. Comments in the open textboxes will be analysed by two members of the working group by comparing the content to existing survey topics and noting those relevant to care, quality of life and symptoms. Findings from the open textboxes will be reviewed by the advisory committee to identify new survey topics or those needing modifications. The criteria for inclusion will be based on the relevance, novelty and value of the topic to the study.
Second round
The second round will include topics within the focus areas that were prioritised by participants in the first round of the study. Within this round, participants will be asked to select a predetermined number of topics from each of the care, quality of life and symptoms categories. The number of topics selected for each category will be determined between rounds one and two, and a weighting calculation will be used to balance the number of topics in the care, quality of life and symptom categories. The topics most selected by participants will move to the next round. Cronbach’s alpha will be used to measure the internal consistency of responses in each round. Free-text comments will be analysed by two members of the working group, for relevant changes to survey topics and reported to the advising group who will decide to add or modify these topics where needed.
Third round
The purpose of the third round will be to receive open-ended feedback on the survey topics that have reached consensus and resolve any uncertainty over survey topics within the care, quality of life and symptom focus areas identified in previous rounds. Suggestions for modifications will be analysed by two members of the working group to inform any final adjustments where needed. These changes will be reviewed and finalised by the Delphi advising committee. The working group will use these findings to develop the minimum data set of questions to inform LC patient registries.
Discussion
The strength of this study stems from its diverse sampling, recruitment and patient-oriented approach. First, the environmental scan captures a broad range of topics relevant to a diverse group of knowledge users across multiple jurisdictions. Second, our recruitment strategy leverages the LCW network, providing diverse perspectives. Third, the study has been designed to ensure that the process is patient-centred, reducing the risk of attrition and improving the interpretability of findings.
Limitations relate to the scope of consensus, language and generalisability. First, the minimum data set produced by this study is intended to reach consensus only for the survey topics within the care, quality of life and symptom focus areas; and not for the selection or wording of any individual questions. Second, the scan of survey questions in the literature was limited to the English-language and the minimum data set produced by this study will also be limited to the English-language. Third, concerning the generalisability of this study, PWLE within an online research community may have LC clinical presentations that vary from the general LC population. We recognise that despite best efforts our participant pool may not be a representative sampling of the LC population that encompasses all lived experiences within our chosen sample size, and that this is a limitation of our study. Finally, given that this study will be conducted with Canadian participants, the results may not be generalisable to other contexts.
We acknowledge that, to date, LC data are already being collected based on the needs of PWLE and researchers within each province, respectively. However, combining efforts to create a minimum data set to standardise data collection in LC research will facilitate advancement in both LC clinical care and research and help to resolve any unmet needs that have yet to be recognised. The results of this study will be especially useful in facilitating data sharing between decision-makers in each province and territory through a national LC patient registry in Canada.
Ethics statements
Patient consent for publication
Acknowledgments
We thank our patient partners, Anne Bhereur, Donna Ellen Frederick and Susan Jolly-Maxie for their contributions to the project and their assistance in creating a patient-friendly survey design. We would also like to thank all of the additional patient partners who contributed throughout the development phase and pilot testing of the survey, and Kathryn Olthius, for her work with the project proposal. Finally, we thank the Long COVID Web Delphi Steering and Advising Committees for their contributions to the design of this study and in their supervision of the development of the Delphi process and Hind Sbihi for their involvement; Tara Brewer and Kristin Dyke, Digital Health, Saskatchewan Health Authority for their contributions to the survey design.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Collaborators Long COVID Web: Anne Bhereur, Caren Cao, Angela Cheung, Simon Décary, Amy Grant, Andrea Gruneir, Gayle Halas, Alison Hoens, Michelle Kho, Grace Lam, Lisa Lix, Kiran Pohar Manhas, Candace McNaughton, Nazeem Muhajarine, Kieran Quinn, Kadia Saint-Onge, Diana Sanchez-Ramirez.
Contributors GG is the principal investigator and guarantor. GG, AAm, PK, HE, AAb, BC, DK, KM, RG, DEF, Long COVID Web and DID are responsible for contributing to the conceptual design of the study. GG, PK and KM are responsible for the supervision of the project. AAm and SS conducted the environmental scan. GG, BC, RG, KM, DID, PK, DEF and Long COVID Web are responsible for revising the survey design. AAm led the protocol planning and prepared the initial draft of the manuscript. RG, KM and BC contributed towards technical writing and editing of the manuscript. All authors had the opportunity to review and revise the manuscript and approve of the final version to be published.
Funding This work was supported by CIHR Canadian Post COVID-19 Condition Research Network Grant #492634 and Long COVID Web Foundational Grant #427761-1084 at the University of Saskatchewan.
Competing interests GG is a member of the Long COVID Web, and has received funding to support the development of this study. In-kind support was received from the Saskatchewan Health Authority, Saskatchewan Centre for Patient-Oriented Research, Saskatchewan Health Quality Council, University of Saskatchewan, George & Fay Yee Centre for Healthcare Innovation and SPOR Support Unit Council. All other authors have no competing interests to declare.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.