Article Text

Download PDFPDF

Original research
Readability and complexity of written information presented to hospitalised patients for trial consent during the COVID-19 pandemic in the UK: a retrospective document analysis
  1. Ewan Gourlay1,2,
  2. Tim Felton2,3,
  3. Mona Bafadhel4,
  4. Christopher E Brightling5,
  5. Jane C Davies6,7,
  6. Rachael A Evans8,
  7. Ling Pei Ho9,10,
  8. Stefan J Marciniak11,
  9. Nick A Maskell12,13,
  10. Joanna Porter14,15,
  11. Elizabeth Sapey16,
  12. Salman Siddiqui17,
  13. Samantha Walker18,
  14. Tom Wilkinson19,
  15. Alex Robert Horsley2,3
  1. 1 North West Lung Centre, Manchester University NHS Foundation Trust, Manchester, UK
  2. 2 Division of Immunology, Immunity to Infection and Respiratory Medicine, The University of Manchester, Manchester, UK
  3. 3 Respiratory Medicine, Manchester University NHS Foundation Trust, Manchester, UK
  4. 4 King’s Centre for Lung Health, King’s College London, London, UK
  5. 5 Institute of Lung Health, University of Leicester, Leicester, UK
  6. 6 Gene Therapy, Imperial College, London, UK
  7. 7 Royal Brompton Hospital, London, UK
  8. 8 University of Leicester, Leicester, UK
  9. 9 MRC Human Immunology Unit, Weatherall Institute of Molecular Medicine, Oxford, UK
  10. 10 Oxford Centre for Respiratory Medicine, Churchill Hospital, Oxford, UK
  11. 11 Medicine, University of Cambridge, Cambridge, UK
  12. 12 North Bristol Lung Centre, Southmead Hospital, Bristol, UK
  13. 13 Academic Respiratory Unit, Department of Clinical Sciences, Bristol University, Bristol, UK
  14. 14 Centre for Inflammation & Tissue Repair, University College London Division of Medicine, London, UK
  15. 15 Interstitial Lung Disease Service, University College London Hospitals NHS Foundation Trust, London, UK
  16. 16 Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
  17. 17 National Heart and Lung Institute, Imperial College London, London, UK
  18. 18 Research & Innovation, Asthma & Lung UK, London, UK
  19. 19 Clinical and Experimental Medicine, University of Southampton, Southampton, UK
  1. Correspondence to Dr Ewan Gourlay; ewangourlay{at}nhs.net

Abstract

Objectives Patient information sheets (PISs) and informed consent forms (ICFs) are essential tools to communicate and document informed consent for clinical trial participation. These documents need to be easily understandable, especially when used to take informed consent from acutely unwell patients. Health literacy guidance recommends written information should be at a level between reading ages 9–11. We aimed to assess the readability and complexity of PISs/ICFs used for clinical trials of acute therapies during the COVID-19 pandemic.

Design Retrospective document analysis.

Setting PISs/ICFs used in trials involving pharmaceutical interventions recruiting hospitalised patients with COVID-19 during the first year of the pandemic were sourced from hospitals across the UK.

Primary and secondary outcome measures PISs/ICFs were assessed for length, approximate reading time and subsection content. Readability and language complexity were assessed using Flesch-Kincaid Grade Level (FKGL) (range 1–18; higher is more complex), Gunning-Fog (GFOG) (range 1–20; higher is more complex) and Flesch Reading Ease Score (FRES) (range 0–100; below 60 is ‘difficult’ for comprehension).

Results 13 documents were analysed with a median length of 5139 words (range 1559–7026), equating to a median reading time of 21.4 min (range 6.5–29.3 min) at 240 words per minute. Median FKGL was 9.8 (9.1–10.8), GFOG 11.8 (10.4–13) and FRES was 54.6 (47.0–58.3). All documents were classified as ‘difficult’ for comprehension and had a reading age of 14 years old or higher.

Conclusions All PISs/ICFs analysed contained literary complexity beyond both recommendations and the reading level of many in the UK population. Researchers should seek to improve communications to improve trial volunteer comprehension and recruitment.

  • COVID-19
  • Clinical Trial
  • Lung Diseases
  • MEDICAL ETHICS

Data availability statement

Data are available on reasonable request. All documents and analysis data are available on request from EG (Orcid: https://orcid.org/0000-0002-0905-8247).

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • This study analyses patient information sheets/informed consent forms (PISs/ICFs) content and readability using commonly used and objective methods.

  • It uses transparent and identifiable results at individual trial level.

  • As a retrospective synthetic analysis, it cannot assess actual patient understanding and opinions at the time of illness/recruitment.

  • PIS/ICF components such as diagrams and other visual elements were not analysed.

Introduction

The COVID-19 pandemic of 2020 saw research trials set up rapidly to be delivered in acute care settings, often by staff with limited prior trials experience and involving acutely unwell patients. This created new challenges for trials staff and patients.1 The suitability of the consent processes has been identified as one of the main concerns around the research response to the pandemic.2 Patient information sheet (PISs) and informed consent forms (ICFs) were adapted from conventional trial templates, following standard UK Medicines and Healthcare Products Regulatory Agency (MHRA) recommendations.3 Trials staff identified that PISs/ICFs were not suited to the circumstances in which consent and trial discussions actually took place, typically involving staff in masks and personal protective equipment (PPE) and patients who were very unwell, afraid and isolated from family.2

Informed consent is both a legal and ethical requirement for healthcare research4 and is a cornerstone of Good Clinical Practice.5 For trials involving investigational medicinal products (IMPs), there is a legal requirement for written consent to be obtained, unless not physically possible and specifically exempted.6 UK and international guidelines state that the written information provided to participants should support conversations around consent, rather than being the sole source of information.3 This is usually provided in the form of a PIS/ICF, often the only written record provided to patients involved in clinical trials.

Increasing length and language complexity of consent forms may correlate with decreasing participant comprehension.7 8 Despite this, PISs/ICFs have become longer and more complex over time,9 reflecting the requirements of sponsors, ethical review boards and/or recommendations from patients. The implementation of European Union General Data Protection Regulations has further extended documents.10 Additional considerations regarding consent for samples stored for future use and for genetic testing have added further complexity.11 A consequence of this is that after recruitment, clinical trial participants often do not retain an understanding of key components of studies.12 13

In response to the issues raised by trials staff delivering studies in acute settings during the pandemic, we were interested to review the information provided to trials participants to help understand the problem and identify whether there are better ways to deliver this. The objectives of this study were to describe the written information presented to participants in acute COVID-19 trials in terms of length, content and readability.

Methods

PISs and ICFs for acute inpatient COVID-19 trials were requested from trials teams across NIHR Respiratory Translational Research Collaboration UK sites via email. PISs/ICFs were reviewed by the lead author to ensure only trials involving IMPs administered to hospital inpatients were included. PISs/ICFs involving personal legal representative, professional legal representative or deferred consent, such as for patients lacking capacity, were excluded. Documents were excluded if published after March 2021. Trials teams were further contacted via email to clarify availability of translated documents, acceptability of telephone translation and availability of online supplemental information sources during the screening period.

Supplemental material

All PISs/ICFs were received as Microsoft Word .doc or .docx files. Total page count was recorded, then documents were prepared for further analysis in line with guidelines for conducting readability analyses by the Centres for Medicare and Medicaid Services14 (full details in online supplemental file).

Document length was analysed via total word count. Approximate time-to-read was calculated based on an average reading speed of 240 words per minute (wpm), as well as upper and lower ranges (175–300 wpm). Reading speed estimates are based on analysis by Brysbaert for adults reading silently.15 PIS subsections were reviewed and recorded. Subsections covering privacy and information governance, benefits of participation and risks of participation were individually analysed by total word count.

Language complexity was analysed using the Flesch-Kincaid Grade Level (FKGL) assessment. This metric assesses both word and sentence length and is expressed as a score (range 1–18; higher is more complex), equivalent to US school grade level reading age (table 1).16 Readability was assessed using the Flesch Reading Ease Score (FRES) (range 0–100; below 60 is ‘difficult’) and the Gunning-Fog Index (GFOG) (range 1–20; higher is more complex). Both metrics assess readability by analysing word complexity and sentence length. There are over 200 readability metrics, with no definitive best metric for either general or health literature.17 These three metrics were selected as they are long established and commonly used, allowing comparison with a large number of previous studies,18 including recent relevant research such as Emanuel and Boyle19 and O’Sullivan et al.10

Table 1

Readability scores with equivalent age, English and US school grades18 35

Documents were also assessed against the objectively assessable plain English recommendations for written medical information.20 These consisted of recommended sentence length 15–20 words; no more than 10% of writing in passive tense; avoiding writing headings in all capitals; avoiding underlining; appropriate use of bullet points.

Analysis software

Total pages, total word count, subsection word count, sentence length and passive tense percentage were calculated using Microsoft Word (Mac v.16.69.1, performed in June 2023). FKGL, FRES and GFOG were calculated using Readable (Added Bytes, Horsham, UK, accessed June 2023)21 by uploading each prepared document following preparation.

Statistical analyses are descriptive. Data were tested for normality using the Shapiro-Wilk test in GraphPad Prism V.9.0 for Macintosh (GraphPad Software, www.graphpad.com) and presented as median (IQR).

Public and patient involvement

Patients and the public were not involved in the design of this study.

Results

We have included information sheets from thirteen clinical trials meeting inclusion criteria. Three supplied PISs/ICFs were excluded (supplement). Of the analysed documents, six were phase 2 trials, three were commercial, single-agent phase 3 studies and four were phase 3 platform studies (supplement). Four studies offered trial summary sheets for patients to read prior to the longer PIS/ICF. Translated PISs/ICFs were available for the RECOVERY trial PIS in 2020, though REMAP-CAP also introduced translated documents for UK use in 2021.

Document length

Median (range) word count for the analysed combined PIS and ICF documents was 5139 words (range 1559–7026) (table 2). For a participant with a mean average reading speed (240 wpm) this length equates to a reading time of 21.4 min (range 6.5–29.3 min). Participants reading at the lower bound reading speed (175 wpm) would take on average 29.4 min (range 8.9–40.1 min). Unlike the other trials included in this analysis, the RECOVERY trial PIS/ICF contained only generic text on IMP risks and was, therefore, notably shorter in length.

Table 2

Summary of document length and complexity for clinical trial patient information sheets and informed consent forms for clinical trials conducted during acute COVID-19 infection

Subsection content

Subsection analysis revealed marked variation in length (supplement). Information relating to privacy and information governance ranged from 72 to 1159 words in total (reading time 0.3–4.0 min), with a median of 519 words. The section on risks of participation ranged from 92 to 1189 words (reading time 0.4–5.0 min), with a median of 519. All trials featured a short section on patient benefits of the research (median 48, range 0–133 words), except for the ACCORD-2 study.

Language complexity

All information sheets featured notable language complexity, median (range) 9.8 (9.1–10.8), with no document scoring lower than an FKGL of 9, equivalent to that of a 14–15-year-old (table 1).

Readability

None of the included PIS scored above 60 on FRES, with a median (range) score of 54.6 (47.0–58.3). Scores below 60 are considered ‘difficult’ for comprehension, with scores 50–60 equating to 15–17-year-old reading level. Assessing readability using the alternative measure the GFOG, the documents had a median (range) score of 11.7 (10.4–13.0). A score of 11 is equivalent to reading age 16–17 (table 1).

Use of plain English

Three out of 13 trial PIS had an average sentence length greater than 20 words. All trial PISs exceeded the recommended 10% writing in passive tense, with a median (range) of 40% (22%–41%). Three out of 13 trials presented headings in capitals, while 2 out of 13 trials did not use bullet points. 10 trials used underlining. No trial met all five assessed plain English recommendations (Supplement).

Discussion

In this study, we have analysed the information provided to hospitalised patients undergoing recruitment to trials of new therapies for COVID-19. The pandemic was an unprecedented international health emergency, and there was urgent need for potential therapies to be trialled in these settings. Trials staff identified that consent processes were often ill suited to the acute settings in which they were delivered.1 Our results would appear to support these subjective observations. Median length was 5303 words, equivalent to a reading time of 21.4 min, though this is likely an underestimate of the true time patients would require. Reading time estimates are derived from studies in healthy volunteers and do not take into account pauses or re-reading of sections.15 There are no data available quantifying the impact of acute illness states on reading speed or accuracy. In this state, median reading time may be nearer the lower reading speed estimate of 29.6 min.

Long and complex PISs/ICFs are not unique to this clinical scenario, with similar findings for paediatric trials,22 surgical trials23 and trials conducted in emergency departments.24 O’Sullivan et al analysed 176 PISs/ICFs used in the UK and Ireland up to 2019, demonstrating readability metrics similar to those found in this analysis.10 Similar issues were also identified by a US study looking at consent and patient information for COVID-19 vaccine studies delivered in non-acute settings.19 This highlights that these COVID-19 studies had very little adaptation for the acute scenario from standard practice. While the longest and most complex documents were phase II trials, this was not universal, with three phase II trials being below the overall median for both length and four for language complexity. HRA/MHRA guidance requires PISs/ICFs involving IMPs to contain specific information,25 potentially limiting the minimum practical document length. Given the wide range of document word lengths (1559–7026) it is likely that it would be legally possible to substantially reduce most documents analysed.

Literacy skills in the UK vary widely. The UK Government Skills for Life survey in 2011 found that 15% of people in England aged 16–65 had reading ability below adult literacy ‘level 1’, broadly equivalent to a reading age of 12–14 years old. Accordingly, one in six individuals would not attain a grade D-G at General Certificate of Secondary Education (GCSE)-level English and would likely struggle to read a train timetable or a pay slip.26 In contrast, the information provided in the trial PISs had a median complexity equivalent to age 14–15 years and median readability was equivalent to age 16–17 years.

Written PISs are only one part of the consent process, and informed consent also involves a trained health professional delivering the information and contextualising this for participants.25 In normal practice, this is relatively straightforward, involving discussion with the patient, sometimes with family members too, to ensure that they understand the study, including risks and benefits. In the specific context of the COVID-19 pandemic, however, staff were required to wear PPE, including masks and face protectors. While necessary to protect staff, studies have shown that wearing PPE, particularly masks, has a negative effect on speech discrimination and comprehension.27 28 For this reason, providing clear and accessible written information for COVID-19 trials was even more important.

During the early months of the pandemic, there were no approved therapies for COVID-19, beyond best supportive care. Patients in the UK did not have access to the wide range of therapies that were being proposed and offered elsewhere without trial evidence.29 The only way of accessing such treatments was in the context of a trial, and patients may have felt particularly incentivised to take part in trials because of this. This may have mitigated against the impact of information being only partially understood. It does not detract, however, from the need to reflect on this aspect of clinical trials in order to understand how to improve the information provided in acute settings.

Four of the included studies also provided a trial summary sheet for patients, so that they could review the most important points of the trial without reference to the much longer full PIS. No other forms of information were available, and none of the studies made use of video explainers during the first year of the pandemic. The RECOVERY study did offer information for patients online,30 while the TACTIC trials published videos online in late 2021.31 Future trials may consider looking at how to use such resources to better inform patients, and clinical trial templates for acute settings might benefit from being more concise and focused. To support rapid progression of medical knowledge during pandemics or other national emergencies, there might be a case for more streamlined consent processes. For example, a two-stage consent might present the most important points only in the acute presentation, followed by a review of consent and confirmation of participation during convalescence.

This study has only looked at the information provided to patients in written form, and a limitation is that we are unable to assess now how this was viewed and retained by patients at the time. We have also only looked at information provided in English, and the analyses of reading time and complexity all assume that English is the reader’s first language. We are unable to assess how many patients would have had added difficulty because this was not the case. No data are available on how many, if any, patients were excluded by inability to fully comprehend written or verbally provided information. Only the two major platform studies had translated documents available. The RECOVERY trial allowed telephone translations if a patient’s native translation was not available in written form, though no others did. Following correspondence with trial sites, we are aware of sites being unwilling to enrol patients in RECOVERY without endorsed written documents. Murali et al showed Asian, black and mixed ethnic groups were under-represented in UK COVID-19 trials.32 While the causes of this are multifaceted, it is likely that language issues were a contributory factor. We have deliberately not compared these consent forms to those used in other acute settings since the primary focus was to address how consent was delivered during COVID-19 to reflect how we can better manage future pandemics. These findings are, however, likely generalisable to other acute settings. Synthetic reliability metrics have inherent limitations such as not assessing layout or non-text components of documents such as diagrams. Furthermore, documents rewritten with improved readability scores do not always improve participant comprehension.33 Nonetheless, their use has been endorsed to guide improvements in readability of PISs/ICFs following consensus meetings.34

In summary, we have shown that clinical trial participant information sheets are lengthy, take significant time to read under optimal conditions, and regularly exceed recommendations on complexity and language. In acute settings, it is especially important to make the communication of trial information clear and understandable. Patient information and consent were identified as being an area for improvement by clinical trials staff. In collaboration with patients and public contributors, proportionate information sheets for acute settings need to be developed with alternative consenting models considered, which include multistep process where complex information is only delivered when patients are well enough to consider it.

Data availability statement

Data are available on reasonable request. All documents and analysis data are available on request from EG (Orcid: https://orcid.org/0000-0002-0905-8247).

Ethics statements

Patient consent for publication

Ethics approval

This study did not require REC approval in line with HRA guidelines (https://www.hra-decisiontools.org.uk/research/). All PISs/ICFs received for analysis were for trials that had received a positive ethical review.

Acknowledgments

We would like to thank the National Institute for Health and Care Research (NIHR) Manchester Biomedical Research Centre (BRC) (NIHR203308) for support.

References

Footnotes

  • X @REvans_Breathe

  • Contributors Conceptualisation by ARH. Data collection and initial analysis by EG. Manuscript initially drafted by EG and ARH. Significant critical review and further development of manuscript by all authors. All authors approve final copy. ARH is the guarantor of this work.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests MB reports unrestricted research grants from AstraZeneca and Roche and has received honoraria to her institution for speaker's fees from AstraZeneca, Chiesi, Cipla and GlaxoSmithKline. She is a scientific adviser to Albus Health and ProAxsi. CEB reports fees to his institution from AZ, GSK, Novartis, Chiesi, BI, Genentech, Roche, Sanofi, Regeneron, Mologics, 4DPharma, Synairgen, Merck. JCD reports reports grants from CF Trust, CF Foundation, CF Ireland, EPSRC and personal fees Vertex Pharmaceuticals, Boehringer-Ingelheim, Eloxx, Algipharma, Abbvie, Arcturus, Enterprise Therapeutics. RAE reports grants from NIHR/UKRI/Wolfson Foundation; consulting fees from AstraZeneca; lecture honoraria from Boehringer; travel support from Chiesi. LPH reports grants from MRC. SJM reports grants from BLF, MRC, June Hancock Mesothelioma Research, Alpha-1 Foundation and Myrovlytis Trust. NAM reports unrestricted research grants from and sits on paid advisory boards for Rocket Medical Plc and BD. JP reports grants from UKRI, LifeArc and MRC. ES reports reports grant funding from NIHR, MRC, HDR-UK, Innovate UK, British Lung Foundation and Alpha 1 Foundation. SS reports speaker fees from GSK, AstraZeneca, Chiesi, Boehringer Ingelheim and Novartis; participates on advisory boards for GSK, AstraZeneca, Chiesi, Boehringer Ingelheim, Novartis, Knopp Biotech, Munipharma, ERT Medical and Owlstone Medical; is a member of the European Respiratory Society Science Council and the UK Medical Research Council; and is a cofounder of Eupnoos Ltd. TW reports grants from UKRI, Synairgen, AZ, UCB, Bergenbio and personal fees from Synairgen and Valneva. ARH reports grants from JP Moulton Charity, CF Trust, CF Foundation, MRC and UKRI and personal fees from Vertex Pharmaceuticals and Mylan Pharmaceuticals.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.