Article Text
PDF
Abstract
Introduction Cystoscopy has revolutionised the process of diagnosing bladder cancer leading to better categorisation of risk levels and more precise treatment plans. Nonetheless, concerns arise about the lack of uniformity among observers in predicting tumour stage and grade. To address these concerns, artificial intelligence (AI) is being incorporated into clinical settings to aid in the analysis of diagnostic and therapeutic images. The subsequent report outlines a systematic review and meta-analysis protocol aimed at evaluating the effectiveness of AI in predicting bladder cancer based on cystoscopic images.
Methods and analysis Our systematic search will use databases including PubMed, MEDLINE, Embase and Cochrane. The articles published between May 2015 and April 2024 will be eligible for inclusion. For articles to be considered, they must employ AI for analysis of cystoscopic images to identify bladder cancer, present original data and be written in English. The protocol adheres to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocol 2015 checklist. Quality and bias risk across chosen studies will be evaluated using the Quality Assessment of Diagnostic Accuracy Studies-2 score.
Ethics and dissemination Ethical clearance will not be necessary for conducting this systematic review since results will be disseminated through peer-reviewed publications and presentations at both national and international conferences.
PROSPERO registration number CRD42024528345.
- Artificial Intelligence
- Urology
- Diagnostic Imaging
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This protocol adheres to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and will incorporate subgroup and sensitivity analyses to further explore the variability among the studies included.
Artificial intelligence (AI)-specific metrics like the F1-score and precision-recall area under the curve will be used to address limitations inherent in traditional pooled analysis such as the impact of imbalanced classes.
Given the novelty of AI technology in cystoscopy, long-term data regarding its impact on clinical outcomes may be scarce.
Limitations on language and the exclusive use of cystoscopy may result in a limited number of eligible studies for inclusion.
Background
Bladder cancer is the most common cancer of the urinary tract and ranks as the ninth most common cancer worldwide according to the WHO’s 2020 report.1 2 The gold standard for diagnosing bladder cancer remains cystoscopy as advocated by both the National Institute for Health and Care Excellence and the European Association of Urology (EAU) guidelines.3 While white light cystoscopy (WLC) is the conventional method widely employed, it potentially overlooks up to 20% of lesions. Furthermore, a recent systematic review and meta-analysis demonstrated that conventional WLC exhibits low diagnostic sensitivity compared with alternative modalities like blue light cystoscopy (BLC).4 Despite the superior detection rates of bladder cancer associated with these alternative methods recommended by EAU, their adoption remains limited likely due to their higher initial costs and limited availability.4 Transurethral resection of bladder tumour (TURBT) with WLC remains fundamental for diagnosing and treating non-muscle invasive bladder cancer which accounts for around 75% of bladder tumours at the time of diagnosis.5
Artificial intelligence (AI) now boasts a remarkable ability to accurately recognise images. AI offers a promising solution to improve the diagnosis of bladder cancer during cystoscopy.6 AI broadly describes the modelling of intelligent behaviour by use of a computer model.7 Deep learning is a subset of AI which more specifically positions AI within the context of medical imaging.8 Augmented cystoscopy employing deep learning holds promise in also enhancing tumour localisation, intraoperative navigation and surgical resection of bladder cancer during TURBT.9
While AI shows promise in the diagnosis of bladder cancer using cystoscopy, several limitations to the deployment of this technology need to be addressed. Given the cystoscopy imaging data used such as WLC and BLC, studies are strongly encouraged to follow the Checklist for Artificial Intelligence in Medical Imaging (CLAIM).10 This is to encourage the reproducibility of AI models in development and to forward the collaboration of research groups in the external validation of their models. The checklist also provides a focus on the use of radiomic features as well as computer-aided diagnosis of imaging data. In order to be better suited for clinical development, models should be explainable in their decision-making process which may be currently under-reported.11 12
Existing reviews on this topic have provided a robust summary of the feasibility of the application of AI in cystoscopy.13 14 However, further investigation of the reported studies with a goal of clinical deployment should be conducted next. Therefore, this systematic review and meta-analysis seeks to outline the precision of AI in forecasting bladder cancer based on cystoscopic images and evaluate its potential influence on patient clinical outcomes.
Methods and analysis
This systematic review protocol adheres to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 checklist.15 This study has been prospectively registered with the PROSPERO review database and all methodologies detailed here have been established prior to implementation. The statistical analysis will focus on evaluating the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and area under the curve (AUC) associated with the application of AI in detecting bladder cancer during cystoscopy along with its impact on clinical outcomes. These parameters will be derived through comprehensive analysis and thematic synthesis of included studies. Pooled sensitivities and specificities across the studies will be determined before calculating PPV and NPV values. AI-specific matrices such as F1-scores and precision recall AUC will also be investigated. This review intends to start on the 1 October 2024 and end on the 30 April 2025.
Search methodology
A comprehensive search will be conducted across multiple databases including PubMed, MEDLINE, Embase and the Cochrane Library. The search strategy will incorporate medical subject heading terms and free text combined with appropriate Boolean operators. Articles from May 2015 to April 2024 will be included to ensure a thorough retrieval of relevant evidence. The search will encompass the following key terms: ‘bladder’, ‘cancer’, ‘diagnosis’, ‘cystoscopy’, ‘cystoscopic images’, ‘artificial intelligence’ and ‘deep learning’. The complete search strategy is outlined in online supplemental file 1. To streamline the initial screening phase, we will be using Rayyan, a semi-automated tool crafted to enhance the efficiency and precision of systematic review.16 All eligible articles identified in the initial search will be imported into Rayyan. Additionally, a manual examination of references cited in all included articles will be conducted to uncover any additional pertinent literature not captured by the initial search strategy. In instances where data is lacking or unclear, corresponding authors will be contacted for clarification.
Supplemental material
Study selection and data extraction
Two researchers, MB and MA, will independently conduct the screening process. They will carefully review the titles and abstracts of eligible studies, eliminating any irrelevant articles. Full-text versions of relevant articles will then be retrieved for further evaluation. In the event of any discrepancies between the researchers, a third reviewer (YZ) will be consulted and a consensus will be reached through discussion. The reasons for excluding articles will be meticulously documented and outlined in a PRISMA flow diagram. Prior to commencing the screening process, calibration exercises will be conducted to ensure consistency among the researchers thereby minimising potential inter-reviewer bias.
Inclusion and exclusion criteria
This systematic review will include studies employing either fully automated or semi-automated AI for analysing cystoscopic images to detect bladder cancer. Both prospective and retrospective studies will be considered. The main comparisons will focus on evaluating sensitivities, specificities, PPVs, NPVs and AUC values. Patient cohorts may include individuals with suspected or confirmed bladder cancer cases with histological findings serving as reference standards.
Excluded from analysis will be correspondence papers, ongoing studies, case reports and conference abstracts. Additionally, non-English language articles, studies not using cystoscopy as the primary diagnostic modality and those involving patients with a history of previous bladder cancer treatment will be excluded.
Data extraction (table of collection)
The data outlined in table 1 will be gathered from all selected studies. Each researcher will independently conduct data extraction, consolidating the obtained information into a comprehensive datasheet. Any discrepancies in data extraction will be reviewed by a third evaluator with the aim of reaching a consensus for resolution.
Data collection items
If accessible, pertinent figures such as true positives, true negatives, false positives, false negatives and their derived calculations will be extracted accordingly. If these figures are not explicitly provided, efforts will be made to compute them from available data. In cases where computation is unattainable, the authors of the respective paper will be contacted to provide the required data.
Endpoints
The main endpoint of analysis will be the statistically significant quantification of accuracy when employing AI in bladder cancer detection during cystoscopy aiming to assess its potential impact on clinical outcomes. Additional outcomes will encompass various parameters examining patient demographics.
Meta-analysis
Should an ample number of suitable studies be accessible, we will proceed with a meta-analysis to amalgamate a quantitative measure of AI performance in identifying bladder cancer from cystoscopic images. Initially, sensitivity and specificity values will be retrieved from studies or if not accessible, computed from clinical data or solicited from authors. If a notable fraction of studies employ alternative metrics like F1-score or precision recall AUC, these metrics will be acquired and scrutinised independently. The distributions of untransformed, logit and double-arcsine transformed proportions will be compared and evaluated for normality using density plots and Shapiro-Wilk tests. The set of ratios most resembling a normal distribution will be selected for further analysis.
Heterogeneity and interstudy variation will be quantified using I2 and if statistically significant, a random effects model will be employed for estimating the summary estimate. Leave-one-out analysis and accompanying diagnostic plots will be used to identify influential studies after the model fits all relevant studies. Summary proportions will be re-estimated based on the remaining studies after each study is removed. Studies exerting a statistically significant influence on the model will be identified as outliers and excluded. After excluding these studies, the model will then be refitted with a summary estimate comprising the remaining studies will be calculated to estimate the accuracy using AI in cystoscopy for bladder cancer detection. All data analysis and visualisation will be conducted using the R statistical environment with the ‘mvmeta’ and ‘meta’ packages.
Risk of bias in individual studies
This study will use the Quality Assessment of Diagnostic Accuracy Studies-2 tool to assess the quality and the risk of bias within each of the included studies.17 The scoring system is split into four main sections: Patient selection, index test, reference standard and flow and timing. Within each section, signalling questions assessed the quality of the research methodology and results at the study level. As included studies use imaging data, the CLAIM-AI checklist will be used to assess imaging specific considerations such as the classification, image reconstruction, text analysis and workflow optimisation.10 Two reviewers (MB and MA) will independently engage in this procedure with any discrepancies resolved through consensus. The bias assessment will analyse the appropriateness and dependability of the data used. This analysis will contribute to evidence synthesis and enhance transparency, potentially leading to the exclusion of articles of low quality or indicative of high bias levels. If included, relevant commentaries will be integrated into the discussion.
Ethics and dissemination
The potential of AI to assist in diagnosing bladder cancer during cystoscopy remains to be thoroughly assessed. Our aim is to consolidate available information to elucidate AI’s role in cystoscopy for bladder cancer diagnosis and outline its impact on clinical practice.
However, it is crucial to acknowledge several limitations. First, given the novelty of AI in this domain, there may be a limited number of studies available for evaluation. Additionally, issues related to the diversity of reporting protocols due to regional variations may arise. Furthermore, the presence of different neural networks could potentially act as confounders due to variations in training data sets leading to discrepancies in reported outcomes.
We intend to disseminate our findings through publication in a peer-reviewed journal. The synthesis of this data will contribute to a deeper understanding of current AI methodologies in bladder cancer diagnosis. Ultimately, this knowledge may facilitate the enhancement of existing systems, both supervised and unsupervised, to refine reporting protocols and improve bladder cancer diagnosis in the future.
Patient and public involvement
There will be no patient or public involvement in this study.
Trial status
Preliminary searches: Started.
Piloting of the study selection process: Not started.
Formal screening: Not started.
Data extraction: Not started.
Risk of bias assessment: Not started.
Data analysis: Not started.
Ethics statements
Patient consent for publication
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
MB and MA are joint first authors.
EB and RH are joint senior authors.
OO and GB contributed equally.
Contributors The authors’ contribution includes but is not limited to the following: MB and MA wrote the manuscript. YZ created the study concept. ML, EB and RH provided supervision and guidance, GB and OO checked the manuscript in its current form. YZ and MB are the guarantors of this work.
Funding Funding for the Open Access publication charges for this article was provided by the Imperial Open Access Fund.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.