Article Text
PDF
Abstract
Introduction Ensuring patient safety in radiation oncology is crucial for delivering high-quality healthcare. Patient safety indicators (PSIs) provide a mechanism for identifying, quantifying and evaluating risks and the effectiveness of safety measures. However, there is currently no specialised set of PSIs tailored for radiation oncology in Germany. This study seeks to: (1) create PSIs specifically designed for radiation oncology settings, (2) develop and psychometrically validate an instrument for assessing safety in German radiation oncology facilities and (3) evaluate the feasibility of implementing this instrument in routine clinical practice. The finalised questionnaire will serve as a self-assessment instrument for radiation oncology departments, aiding them in evaluating their efficacy in ensuring patient safety, prioritising safety interventions and tracking performance over time.
Methods and analysis We are undertaking a 3-year, mixed methods study to address our objectives. For the identification of PSIs, we will conduct a comprehensive review on the PubMed database, along with reviewing national and international guidelines and recommendations. To refine the initial set of indicators, we will consult with experts, including physicians, medical physicists, nurses, administrators and radiation therapists through focus groups. We will employ a Delphi study for the final consensus and selection of indicators. Additionally, the perspectives of patients will be incorporated by formation of a project patient’s committee which meets throughout the project phases. We will reformulate the identified PSIs into questionnaire items. The questionnaire’s clarity and comprehensibility will be validated through cognitive interviews, followed by psychometric testing in a pilot group of over 150 participants from German radiation oncology departments. The final version of the questionnaire will then be implemented in routine healthcare settings and we will interview individual users about their experiences with the questionnaire in semistructured interviews. We will convene a subsequent expert workshop to discuss the study results and explore avenues for the questionnaire’s broader implementation. The finalised questionnaire will be made accessible via a web app. We hereby present the study potocol as a pre-results report.
Ethics and dissemination Ethical approval for this study was granted by the Hamburg Ethics Committee (Approval Number: 2023-101018-BO-ff). This trial is registered by the ARO (Arbeitsgemeinschaft Radioonkologie /working group for radiation oncology of the German Cancer Society), protocol number 2023-03 and in the German register for clinical trials with the number DRKS00034690. Study results will be published in conference papers and talks as well as journal papers with focus on open access journals. The results will be also disseminated during the implementation workshop in phase III, which will involve a diverse group of stakeholders.
Trial registration number DRKS00034690
- Radiation oncology
- Radiation oncology
- Patient Participation
- Patient-Centered Care
- MEDICAL PHYSICS
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study provides the first systematic instrument to assess patient safety in the field of radiation oncology in Germany.
The study conception includes patient’s views by implementation of a patient’s advisory board as well as evaluation of patients interviews on patient safety.
The developed instrument provides a questionnaire which underwent psychometric evaluation across multiple institutions.
One limitation of this project is that no data comparison is planned for quantitative metrics against the degree of implementation of safety indicators.
Results of focus groups and interviews strongly depend on participants and are of qualitative nature. Yet, the study design, which combines qualitative sources with a literature review and ratings by a large board of experts, should minimise these effects.
Background
To ensure the therapeutic success of contemporary radiotherapy modalities, consistent quality across all procedural steps is imperative.1 2 Mistakes in therapy planning and execution can result in adverse events, including patient fatalities.3–5 In Germany, akin to other European nations, the Radiation Protection Act and Ordinance stipulate guidelines for thresholds and quality assurance measures concerning the use of ionising radiation in medical applications. These regulations mandate prospective risk assessments and the reporting of critical radiation-related events. However, the exact methodologies for effective risk management can vary significantly across individual facilities.6 Moreover, the absence of centralised guidelines necessitates considerable effort to establish facility-specific safety protocols.
Epidemiological data, primarily based on reported adverse events, likely underestimate the true risk to patient safety across healthcare settings.2 Additional methods such as monitoring by indicators are therefore favoured.7 These are designed to identify and quantify situations most likely to result in adverse events. While various sets of patient safety indicators (PSIs) are available for general healthcare, those are difficult to adapt for radiation oncology. Existing safety indicators in the healthcare sector often focus on inpatient scenarios like fall or pressure ulcer prevention, which are not directly applicable to the predominantly outpatient-based radiation oncology treatments.8 Oncological treatments, including radiotherapy, entail unique risks that demand specialised expertise for effective risk assessment and the identification of potential therapy side effects. For radiation oncology, several groups reported on the related topic of quality indicators, which focuses not only on effectivity, effectiveness and reproducibility but also on safety of radiation therapy. Nevertheless, none of those encompass all aspects of patient care including supportive care and medication alongside with safety culture, risk management and aspects of personal resources.9–12
Objectives
The primary objectives of this study are: (1) to develop PSIs tailored specifically for radiation oncology, (2) to design and psychometrically validate an instrument for assessing patient safety in German radiation oncology facilities and (3) to evaluate the feasibility of implementing this instrument in routine clinical practice.
Methods
This investigation is a 3-year, mixed methods study, structured into three distinct phases. An illustrative overview of the study design is provided in figure 1. The study commenced in July 2023 and is planned to conclude in June 2026, with each project phase lasting approximately 1 year. A detailed overview of the aimed schedule is available in the study synopsis in online supplemental material.
Supplemental material
Overview of study design and samples. *Experts include all five relevant professional groups: Physicians, medical physicists, radiation therapists (RTTs), nurses and administrative staff.
Phase I: identification of PSIs
In this initial phase, we aim to identify PSIs pertinent to radiation oncology, with an emphasis on the quality of processes and structures and, moreover, on possible quantitative data representing benchmarks associated with safe patient care. An extended panel of experts will discuss these indicators, initially in focus groups, culminating in a consensus through a Delphi study.
A comprehensive review for relevant PSIs in radiation oncology will be conducted using PubMed as well as national and international recommendations from affiliated societies. We will include all publications from 1990 until today in German or English which cover relevant keywords concerning radiation oncology and patient safety. Data from incident reporting systems as well as from published risk analyses are also included in our screening if they underline certain risks specifically preventable through safety measures. Further, we will include papers on oncology and systemic tumour therapies, which cover both patient safety and supportive care. Publications for in vitro data or animal models as well as publications covering only specific therapy options (like proton therapy) will be excluded. Examples of suitable PSIs include measures in the treatment process (eg, patient identification), compliance with legal requirements for machine-based quality assurance (eg, calibration intervals) and adherence to dose limits. Existing PSIs from known questionnaires are also considered. English-language PSIs identified in the literature search are translated following the TRAPD (translation, review, adjudication, pretest and documentation) protocol.13 The objective is to capture indicators as broadly as possible. Based on the literature review, we will also compile different dimensions of patient safety specific to radiotherapy.
To refine the initial set of indicators, experts will participate in focus groups where they discuss the relevance of identified PSIs and can propose additional ones. The focus group discussions aim to streamline the total number of indicators for clarity. These sessions will be audio recorded, transcribed and anonymised. We will analyse them using qualitative content analysis, employing both deductive and inductive approaches.14
Further, we will conduct semistructured interviews with patients who are receiving or have received radiotherapy. In those interviews, we will discuss all PSIs that concern interactions with patients. We will discuss the patients’ experiences with patient safety throughout their radiotherapy and will explore whether they want to add any PSIs. Again, the interviews will be audio recorded, transcribed, anonymised and analysed using qualitative content analysis.14
In order to design the instrument efficiently, the time frame and effort required to use it must be limited, which requires a selection of indicators. This will be achieved through a two-round Delphi study15 using an online survey administered via LimeSurvey (LimeSurvey GmbH, Hamburg, Germany). Participants will rate the relevance and accessibility of the indicators and a free-text field will allow for the addition of other relevant indicators. In the second round, participants will receive feedback on the first round’s results and have the opportunity to revise their ratings.15
Sample
Given that radiation oncology is an interdisciplinary field, our panel of experts will comprise five professional groups: physicians, medical physicists, radiation therapists, nurses and administrative staff. Participants will be recruited via email lists from professional societies, supported by collaborating departments. Each professional group will be represented by at least two participants. Two focus groups, each consisting of 8–10 participants, will be conducted, which include 8–12 patients covering different ages, different diseases and types of therapy indication (palliative vs curative vs prophylactic). The Delphi study will be carried out in two rounds with a minimum of 10 participants who will be recruited through email, supported by professional societies and partner institutions.
Phase II: psychometric testing
The PSIs selected through the Delphi process will be transformed into questionnaire items, which will be evaluated using a scale (eg, degree of agreement). By categorising these safety indicators under their respective dimensions, a structured questionnaire with subsections will be developed.
The questionnaire’s comprehensibility will be initially assessed through cognitive interviews.16 These interviews will be audio recorded, transcribed, anonymised and subsequently analysed. If item revisions are required, the process will be repeated until no changes are necessary.
The finalised questionnaire will then be pilot-tested among the participating professional groups at collaborating institutions, using an online survey administered via LimeSurvey (LimeSurvey GmbH, Hamburg, Germany).
Based on the gathered data, we will perform psychometric analyses. Item difficulty will be calculated to examine ceiling and floor effects, thereby ensuring sufficient variability in the responses.17 Missing values will serve as a proxy for item acceptability. Inter-item and item-total correlations will be assessed to eliminate redundancy. Structural validity will be confirmed through factor analysis, modelling the dimensions identified in phase I. Model fit will be evaluated based on several fit indices including root mean square error of approximation, standardised root mean square residual, Tucker-Lewis index and comparative fit index.18 The final questionnaire items will be selected in a team discussion, considering both psychometric properties and content relevance.
Sample
Participants for the interviews will again represent all five professional groups involved in patient care and will be recruited similarly to phase I. Each questionnaire item will be tested in at least five cognitive interviews per round. The sample size will vary depending on the questionnaire’s size and the number of testing rounds (eg, n=5–10). For the psychometric evaluation, a sample size of 3–5 persons per question is deemed suitable, although the exact number of questions will be determined in phase I. An average of three participating institutions, each with an average of three staff members, is planned for each of the 17 state medical associations (‘Ärztekammer Kreis’) (n=153). Recruitment will again be conducted via email, supported by professional societies and affiliated institutions.
Phase III: test for feasibility in routine care
The final questionnaire will be introduced into routine healthcare settings for feasibility testing. Participating facilities will have the opportunity to self-evaluate using the indicators, calculate their scores and, where necessary, identify measures to enhance patient safety. Comprehensive support will be extended to the facilities to determine the most effective means of integrating the questionnaire into their existing workflows and structures. Options include using a web app or paper formats, along with guidelines specifying who should complete the questionnaire and when.
To gain insights into user experiences for the experts of the phase before, semistructured interviews will be conducted with individual users. The focus will be on determining factors that either facilitate or hinder the effective use of the questionnaire and the application of its results. The interview guide will be structured around established implementation frameworks, such as the Consolidated Framework For Implementation Research.19
Subsequently, an expert workshop will be held to discuss the findings and explore options for broader implementation and utilisation of the questionnaire, as well as future studies comparing PSI with clinical metrics and outcome.
Web app development
Concurrent with phases II and III, an external provider will design and develop a web app to host the questionnaire. Once launched, users will have immediate access to category-specific evaluations. Optionally, the collected data can be shared with the project team or future users for further scientific exploration.
Sample
The final testing phase will involve at least four healthcare facilities. Within each facility, a minimum of three interviews will be conducted across different professional groups. An expert workshop will gather participants from across Germany, including professionals in radiotherapy, patient representatives and other experts, amounting to approximately 30 participants in total.
Patient and public involvement
We will involve patients by forming a patients’ advisory board, which holds multiple meetings throughout the project (at least two per year, more if necessary). Four patients of the university hospital have agreed to advise the project. Possible tasks in general are discussions of the current state of the project and in particular advisements for the semistructured interviews with patients. Here, the patient advisory board can identify ways to recruit patients for the interviews, elaborate the interview guide to ensure comprehensibility of the task and relevance for the patients as well as contribute to the interpretation of the results. Furthermore, the patient advisory board will contribute to the dissemination of information about the project. Here, the patients will assist in writing and revising lay information about our project which will be available online. Finally, the patient advisory board will be invited to our expert workshop and offered the possibility to report their own work and experiences within the project.
Discussion
This study aims to create the first instrument specifically designed to systematically assess patient safety in the field of radiation oncology in Germany. Beyond an extensive literature review, the study incorporates expert recommendations from all pertinent professional groups. The formation of a patient’s advisory board offers the opportunity to include the patient’s point of view through direct involvement. Additionally, the developed questionnaire will undergo psychometric evaluation across multiple institutions. The study will also investigate potential barriers and facilitators for its implementation through interviews and an expert workshop.
The finalised questionnaire will serve as a digital self-assessment tool for radiation oncology departments. This tool will enable these departments to evaluate their performance in various facets of patient safety, prioritise interventions to increase patient safety and track outcomes over time. The final tool can serve to monitor and improve patient safety on a broad basis in conjunction with incident reporting and patient follow-up data. One limitation of this project is that no data comparison is planned for these quantitative metrics against the degree of implementation of safety indicators. Subsequent projects might show quantitative improvements through comparing those metrics against each other.
Moreover, this initiative contributes to the broader discourse on safety awareness within healthcare. It provides individual facilities the opportunity to benchmark their processes against national and international guidelines, thereby fostering a culture of continuous improvement in patient safety.
Ethics and dissemination
This study will adhere to the most recent version of the World Medical Association's Declaration of Helsinki, upholding the principles of good scientific practice. Participation in the study is entirely voluntary and poses no foreseeable risks to the participants. All participants will be fully informed of the study's objectives, data collection procedures and the intended use of the collected data. Prior to their involvement, written informed consent will be obtained from each participant. Rigorous measures will be implemented to ensure compliance with data sensitivity, data protection and confidentiality standards. Ethical approval for this study has been granted by the Hamburg Ethics Committee (Approval Number: 2023-101018-BO-ff).
To maintain transparency and engagement, a biannual newsletter will be distributed to study participants, collaboration partners and the interested public, updating them on the study’s progress and key findings. The study’s outcomes will be submitted for publication in scientific journals, with a preference for open access formats to maximise accessibility.
In addition to scholarly publication, the results will be disseminated during the implementation workshop in phase III, which will involve a diverse group of stakeholders. Moreover, findings will be showcased at relevant national and international conferences to broaden their impact within the professional community.
The questionnaire developed through this study will be made available online, ensuring its universal accessibility and paving the way for its widespread adoption in improving patient safety in radiation oncology.
Ethics statements
Patient consent for publication
Acknowledgments
The authors would like to thank the associated institutions for support during study planning and recruitment: PD Dr Cihan Gani, Tübingen; Professor Dr. Hans Christiansen, Hannover; Professor Dr. Hans-Theodor Eich, Münster; Professor K. A. Hartmann, Düsseldorf; PD Dr. med. David Kaul, Berlin; PD Dr. med. Gunther Klautke, Chemnitz; Professor Dr. Heinz Schmidtberger, Mainz; Professor Dr Dirk Vordermark, Halle; Professor Dr. Andrea Wittig-Sauerwein, Würzburg; Professor Dr. Florian Würschmidt, Hamburg; Dr. med. Christina Woywod, Pinneberg; PD Dr. Jan Kriz, Münster; Christiana Lütter, Köln; Professor Dr. Ursula Nestle, Mönchengladbach; Dr. Laith Samhouri, Lüdenscheid; Professor Dr. Markus Buchgeister, DGMP; Angelika Kleine, OVIRO; Gunthard Kissinger, M.U.N.D.; Helmuth Wolken, Selbsthilfe Lungenkrebs Hamburg; Birgit Schlömp, VMTRO; Silke Wischer, BVPRO; Dr. Michael Baehr, UKE Klinikapotheke; Avin Hell, UKE UCCH Patient:innenbeirat; and Professor Dr. Levente Kriston, UKE Hamburg.
We acknowledge financial support from the Open Access Publication Fund of UKE - Universitätsklinikum Hamburg-Eppendorf.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
AB and EC contributed equally.
Contributors AB, EC and MG wrote the manuscript. AB and IS designed the study with input from EC and MG. AB is responsible for the overall content (as guarantor).
Funding The study is funded by the German Cancer Aid (Deutsche Krebshilfe, Buschstr. 3253113 Bonn), grant numbers 70115408 and 70115490.
Competing interests MG receives travel support from LAP GmbH Laser Applikationen. All other authors declare no competing interests.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.