You are here

  • Home
  • Archive
  • Volume 12, Issue 5
  • Development and validation of a clinical diagnostic model for pregnant women with renal colic in the emergency department in China: a protocol for a retrospective cohort study

Article Text

Article menu

Download PDFPDF

Obstetrics and gynaecology
Protocol
Development and validation of a clinical diagnostic model for pregnant women with renal colic in the emergency department in China: a protocol for a retrospective cohort study
  1. YuZhan Lin1,
  2. ZhiKai Xu2,
  3. XiangCui Ding3,
  4. Lei Chen1,
  5. KangWei Dai1
  1. 1 Department of Clinical Laboratory, Ruian People's Hospital, Ruian, China
  2. 2 Department of Ultrasound Imaging, Ruian People’s Hospital, Ruian, China
  3. 3 Gynecology Department, Ruian People’s Hospital, Ruian, China
  1. Correspondence to Mr YuZhan Lin; 1016099047{at}qq.com

Abstract

Introduction Urolithiasis affects many people throughout their lives. Among the maternal population, although the morbidity of acute urolithiasis in pregnant women is unremarkable, it is the leading cause of hospitalisation during pregnancy. There is no effective clinical diagnostic tool to help doctors diagnose diseases. Our primary aim was to develop and validate a clinical prediction model based on statistical methods to predict the probability of having disease in pregnant women who visited the emergency department because of urolithiasis-induced colic.

Methods and analysis We will use multivariate logistic regression analysis to build a multivariate regression linear model. A receiver operating characteristic curve plot and calibration plot will be used to measure the discrimination value and calibration value of the model, respectively. We will also use least absolute shrinkage and selection operator regression analysis combined with logistic regression analysis to select predictors and construct the multivariate regression model. The model will be simplified to an application that has been reported before, and users will only need to enter their clinical parameters so that risk probability is automatically derived.

Ethics and dissemination The review and approval documents of the clinical research ethics committee have been received from the ethics committee of our hospital (The Third Affiliated Hospital of Wenzhou Medical University). We will disseminate research findings through presentations at scientific conferences and publication in peer-reviewed journals.

  • Urolithiasis
  • Urogynaecology
  • GYNAECOLOGY
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi-org.ezproxy.u-pec.fr/10.1136/bmjopen-2021-056510

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This statistical model will be the first prediction model based on data from pregnant women with symptomatic stones, and this protocol will use the PROGRESS framework and follow Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis guideline.

    • This prediction model will be developed by systematically reviewing existing literature and referring to reported prediction model literature to meet clinical needs and to ensure consistency with existing clinical guidelines.

    • This study will be based on relevant literature, including a systematic review of existing prediction modelling and will select a broad series of relevant candidate predictors.

    • We will use least absolute shrinkage and selection operator regression analysis combined with logistic regression analysis to select predictors and construct the multivariate regression model.

    • It will lose partial useful data that can adversely affecting the results of the study due to such retrospective cohort study as clinical data were collected from previous electronic medical records.

    Introduction

    Urolithiasis affects many people throughout their lives. Recently, the trend of urolithiasis has increased, and there has been corresponding increases in outpatient visits and the financial cost of treating the disease.1 2 Diet, climate changes and some metabolic diseases such as diabetes and hypertension will lead to urolithiasis.3 Especially, this trend of urolithiasis has become more prominent in women than in men, whereas previously it was more frequent in men. The National Health and Nutrition Examination Survey database has reported that urolithiasis rates in men and women were 10.6% and 7.1%, respectively, and it was a prominent disease in middle-aged men.1 4 In the pregnant population, although the morbidity of acute urolithiasis is unremarkable, it still is the major reason for non-obstetric hospital admissions.3

    Urolithiasis in pregnancy is an infrequent disease, but it can lead to many poor deliveries. Urolithiasis can cause premature rupture of the membranes, preterm labour and preterm birth in pregnant women who have symptomatic urolithiasis.3 5 6 The limitations of imaging methods, laboratory tests and clinical treatments make it difficult for emergency physicians to determine the treatment options, which will lead to misdiagnosis and potential risks of both the mother and fetus.5 7 8

    In recent years, the multivariable clinical prediction model (CPM) has become increasingly popular in diseases diagnosis.9–12 As a single predictor is always insufficient to measure a valuable HR of disease, especially in some complicated situations, the CPM can combine multiple predictors to predict the incidence of related diseases or to classify the level of diseases, and it can indicate effective treatment for patients.12 13

    The current diagnostic methods of urolithiasis in pregnant women are based on the subjective judgement of the physician/internist when ultrasonography cannot clearly show urolithiasis and radiological imaging methods are limited, because urolithiasis can lead to pregnancy loss or foetal anomalies.14 There is no effective clinical diagnostic tool to help doctors diagnose diseases. Even so, the CPM is still uncommonly used in clinical diagnosis because of its complicated operation or lack of credibility in diagnosing diseases.9 To resolve these issues, we will simplify the CPM by referring to the method published by Okita et al,12 verifying the sensitivity and specificity of the model, and conducting internal validation of the model.

    Objectives

    Primary objective

    The primary objective of this study was to develop and internally validate a clinical diagnostic model that can be used not only to detect potential urolithiasis in pregnant women on a daily basis but also to help diagnose flank colic in pregnant women suspected of having urolithiasis in the emergency department.

    Secondary objectives

    We plan to conduct an epidemiological survey to analyse the incidence of urolithiasis in pregnant women in recent years. The epidemiological investigation will include the use of statistical methods to describe the demographic, clinical signs, imaging data and laboratory data of patients. We will compare the clinical data of healthy pregnant women with those of urolithiasis-positive pregnant women to identify additional risk factors that are strongly associated with urolithiasis.

    Methods and analyses

    Ethics statements

    This study will be reported according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement, which is used to present studies concisely and transparently.15 We will use novel measurement methods introduced by Steyerberg et al and cited in many studies to verify the performance of the prediction model.9 16

    Source of data and patient selection

    This study will be a retrospective cohort study. The baseline data of patients will come from the database of HaiTai electronic medical records (Nanjing Haitai Medical Information System Co, Nanjing, China) at our hospital. First, we will screen patients who have been treated in the emergency department of our hospital from January 2009 to December 2019 for lateral abdominal pain or colic. Patients who have undergone renal and ureteral ultrasonography bilaterally, low-dose CT and MRI at our hospital will be included in our database. Second, we will exclude female patients who do not have results of a prenatal ultrasound examination in the same year. We will obtain all imaging records and allow a radiologist with national certification in health technology and a gynaecologist with a senior professional title access them without knowing the patient information and previous diagnosis; the two experts will not know each other’s diagnoses. Every pregnant woman’s electronic medical record will be reviewed to determine if symptomatic stones developed during pregnancy, and the integrity of the data will be reviewed. The positive cases will be used to establish the model. Meanwhile, the clinical data collected from negative and positive cases will be compared using t-test/χ2 test to identify novel predictors. Diagnostic criteria for urolithiasis will be based on the Chinese Guidelines for the Diagnosis and Treatment of Urological Diseases (Edited by Chinese Urology Surgery Branch, published by People’s Medical Publishing House V.2014). The name of the diagnosed disease will be based on the International Classification of Diseases, 10th Revision.

    The inclusion criteria for positive patients will be as follows: (1) pregnant woman who were admitted to the hospital because of urinary tract obstruction or renal colic and diagnosed with urolithiasis after examination by two experts (a radiologist and a gynaecologist); and (2) pregnant women with documented clinical signs and laboratory data (all the clinical signs and laboratory data that will be used to diagnose urolithiasis are discussed in the Candidate predictors subsection). The exclusion criterion will be patients who do not meet the aforementioned inclusion criteria.

    Sample size

    To achieve lower bias and better performance of the prediction model, we will refer to the conclusion that Moons and Wolff drew in their paper published in 201917: the larger the sample sizes, the better the results (smaller SEs and narrower CIs). This is more important in prediction models than in other models, and we will also consider the actual number of patients we have received every year. We will set the events per variable to approximately 20, as Ogundimu et al recommended in their article.18 Based on the data we have collected from 2018 to 2019 (60 cases of symptomatic urolithiasis), we conservatively estimate that we will collect more than 500 clinical cases of symptomatic urolithiasis in pregnant women plus 500 confirmed cases of non-urolithiasis or symptomatic urolithiasis during pregnancy, and 1000 eases will be used for model training and validation. The number of patients used for training and validation will be shown in a flowchart.

    Candidate predictors

    We conducted a systematic review to find identified predictors that have been previously reported as risk predictors to assess the potential patients with urolithiasis or to diagnose the probability of urolithiasis when the patient visits the emergency department because of flank pain.5 12 19–21 We have consulted specialists in obstetrics and gynaecology, the imaging department and clinical laboratory for all predictors. Subsequently, we concluded that the following would be candidate predictors:

    1. Presence of gestational diabetes.

    2. Presence of gestational hypertension.

    3. Gestational age.

    4. Presence of urinary irritation symptoms.

    5. Presence of nausea and/or vomiting.

    6. Haematocrit level.

    7. Albumin level.

    8. Uric acid level.

    9. Hypersensitive C reactive protein level.

    10. Presence of haematuria.

    11. Electrolyte levels (including potassium, sodium, chlorine, calcium, phosphorus and magnesium).

    12. Presence of hydronephrosis and side of hydronephrosis.

    13. Ultrasonic examination results and stone location.

    14. History of calculi.

    15. Presence of knocking pain in the area of the kidneys.

    16. Type of pain and Numeric Rating Scale score for pain.

    17. Smoking habit.

    18. Alcoholism.

    19. History of drug use.

    Selection of predictors

    The number of risk predictors needs to be appropriate, and multicollinearity among the predictors, as well as the applicability and ease of use of the final prediction model, needs to be considered. For these reasons, we will use least absolute shrinkage and selection operator (LASSO) regression analysis combined with logistic regression analysis to select predictors and construct the multivariate regression model. The LASSO method has superiority of selection of predictors, and it can manage collinear partial estimation as it adds a penalty term to the least square method to determine which predictors are most relevant to the outcome.22 The outcome of LASSO regression analysis will be presented in a graph. All clinical predictors will be collected from patients’ electronic medical records.

    Data management

    We have developed the following set of rules to improve the effectiveness of the patient data collected.

    1. Positive cases should occur during gestation, neither before nor after gestation.

    2. If patients have several records of visits due to symptomatic urolithiasis, the one with the most quantitative and qualitative data will be selected.

    3. All clinical data during pregnancy will be collected, with variability of the clinical indicators considered.

    4. Any positive signs that are present will be recorded as positive, even if the positive signs disappear later because of medication or other reasons.

    5. Pregnant women with gestational hypertension and gestational diabetes mellitus should have corresponding case histories and be treated with relevant drugs.

    6. All clinical data will be collected before the intervention or medication.

    7. Imaging data of the affected side will be collected.

    Continuous variables will be transformed into category variables according to statistical analysis if necessary. Both types of data will be presented in tables (continuous variables will be presented as mean±SD, and category variables will be presented as number and percentage). To ensure the integrity of the data of patients for the training and validating prediction models and to avoid coding errors, overall clinical data that are relevant to patients will be reviewed by two experienced clinical researchers in a password-protected medical data system. Both researchers have more than 4 years of clinical working experience.

    Missing data

    To solve the inevitable problem of missing values in the collection of patients’ data, we will use multiple imputation, which was based on the chained equation and recommended in many studies9 13 23 24 to resolve the problem of missing data at random. Additionally, we will exclude the data missing more than 80% for effectiveness of multiple imputation.9

    Statistical analysis

    We will use multivariate logistic regression analysis9 24 25 to establish a multivariate regression linear model to predict the risk of symptomatic urolithiasis during pregnancy. The discrimination value and calibration value of the model will be used to evaluate the performance of the model and will be presented as a receiver operating characteristic (ROC) curve plot and calibration plot, respectively. At the same time, we will use decision curve analysis (DCA) and combine the cut-off value obtained from the ROC curve to calculate the clinical net benefit value to analyse the clinical validity of the prediction model and solve the problems caused by misdiagnosis and missed diagnosis.

    Construction of the risk prediction nomogram and model simplification

    We will apply the covariable of each identified risk predictor obtained via logistic regression analysis to establish a diagnostic prediction model. We will refer to Okita et al’s previous idea12 to simplify the model into a mobile phone application. Users will only need to input several required clinical parameters, such as duration of pregnancy (in weeks), laboratory test data, etc, to automatically generate the probability of disease.

    Patient and public involvement

    Neither patients nor the public was involved in setting the research question or the outcome measures, designing the investigation or interpreting the data. There are no plans to involve patients in the dissemination of the results.

    Discussion

    The primary purpose of this study was to construct and validate a prediction model that can be used not only to detect potential urolithiasis in pregnant women on a daily basis but also to help diagnose flank colic in pregnant women suspected of having urolithiasis in the emergency department. Therefore, the positive population must be patients diagnosed with urinary tract stones and with obstructive symptoms. If a patient does not have symptoms of urinary tract obstruction, she will be classified as negative to distinguish whether the cause of renal colic in this patient is urinary tract calculi or other reasons. Although there is no direct relationship between urinary calculi and pregnancy, the model population that will be used in this study is pregnant women, which is of great significance in establishing a diagnostic model.

    To our knowledge, this statistical model will be the first prediction model based on data from pregnant women with symptomatic stones, and this protocol will use the PROGRESS framework and follow the TRIPOD guideline. We will use some common laboratory data and easily observed clinical symptoms as risk predictors to predict disease risk. All candidate predictors used for risk prediction were previously reported in the existing literature3 5 12 19–21 26 to ensure the effectiveness of the model.

    The calculating result of the model will be presented as a percentage, and the corresponding level of risk (low, medium or high) will be presented too. We will use more modern statistical methods (LASSO regression analysis) to screen for risk predictors. Regarding model performance validation, the ROC curve and calibration plot will be used to calculate the model’s discrimination value and calibration value, respectively. Moreover, DCA will be used to estimate the clinical validity of the prediction model and to resolve questions of misdiagnosis and missed diagnosis. The prediction model will be simplified as a mobile application for easy use and popularisation.

    We will lose partial useful data that can adversely affect the study’s results due to the retrospective cohort study design, as the clinical data will be collected from patients’ electronic medical records. Problems in retrospective studies also include recall bias, coding errors and missing data. It is very important to use effective methods to solve the problems of missing values and recall bias to build prediction models.27 In this study, we will use multiple interpolation to solve the problem of missing data. To solve recall bias, we will obtain all the imaging data, and two qualified doctors will review the imaging results and make a blinded diagnosis. There are still too few patient-related clinical indicators included in our study, and more specific maternal molecular markers and imaging-related parameters can be added in the future.

    This prediction model cannot be applied to people from other regions or other races because the training data used in this study was from a single centre. This weakness will need to be compensated for in the future by multicentre, multiracial data.

    In the absence of imaging evidence, the diagnosis of urolithiasis in pregnant women remains controversial. Therefore, the diagnosis and treatment of disease can only rely on the subjective judgement and empirical treatment of clinicians. In this study, we will collect all imaging records and have radiologists with national certification in health technology and gynaecologists with a senior professional title access them and make a final diagnosis without knowing the patient’s previous diagnosis and the diagnosis made by another person.

    The CPM can provide a relatively accurate prediction value for diagnosing or evaluating the stratification of diseases; however, many questions still remain, and all of them should be answered before the CPM can be used in clinical practice.14 The primary question is the determination of ending events. Emergency physicians should pay more attention to the consequences of making a diagnosis of urolithiasis in pregnant women with flank pain when radiology cannot be used.14 28 Additionally, the CPM should not only be confirmed by the statistical assessments but should also be useful in clinical application. Ultimately, the clinician needs to use the CPM and trust it.14 29

    Ethics and dissemination

    The examination and approval documents of the clinical research ethics committee have been received from the ethics committee of our hospital (The Third Affiliated Hospital of Wenzhou Medical University). We will disseminate research findings through presentations at scientific conferences and publication in peer-reviewed journals.

    Ethics statements

    Patient consent for publication

    References

    1. ↵
      2. Semins MJ ,
      3. Matlaga BR
      . Kidney stones during pregnancy. Nat Rev Urol 2014;11:163–8.doi:10.1038/nrurol.2014.17 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/24515090
      OpenUrlPubMed
    2. ↵
      2. Mefford JM ,
      3. Tungate RM ,
      4. Amini L , et al
      . A comparison of urolithiasis in the presence and absence of microscopic hematuria in the emergency department. West J Emerg Med 2017;18:775–9.doi:10.5811/westjem.2017.4.33018 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/28611901
      OpenUrlPubMed
    3. ↵
      2. Semins MJ ,
      3. Matlaga BR
      . Management of urolithiasis in pregnancy. Int J Womens Health 2013;5:599–604.doi:10.2147/IJWH.S51416 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/24109196
      OpenUrlPubMed
    4. ↵
      2. Scales CD ,
      3. Curtis LH ,
      4. Norris RD , et al
      . Changing gender prevalence of stone disease. J Urol 2007;177:979–82.doi:10.1016/j.juro.2006.10.069 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/17296391
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      2. Somani BK ,
      3. Dellis A ,
      4. Liatsikos E , et al
      . Review on diagnosis and management of urolithiasis in pregnancy: an ESUT practical guide for urologists. World J Urol 2017;35:1637–49.doi:10.1007/s00345-017-2037-1 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/28424869
      OpenUrlPubMed
    6. ↵
      2. Swartz MA ,
      3. Lydon-Rochelle MT ,
      4. Simon D , et al
      . Admission for nephrolithiasis in pregnancy and risk of adverse birth outcomes. Obstet Gynecol 2007;109:1099–104.doi:10.1097/01.AOG.0000259941.90919.c0 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/17470589
      OpenUrlCrossRefPubMed
    7. ↵
      2. Wymer K ,
      3. Plunkett BA ,
      4. Park S
      . Urolithiasis in pregnancy: a cost-effectiveness analysis of ureteroscopic management vs ureteral stenting. Am J Obstet Gynecol 2015;213:691.e1–691.e8.doi:10.1016/j.ajog.2015.07.024 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/26215329
      OpenUrlPubMed
    8. ↵
      2. Gottlieb M ,
      3. Long B ,
      4. Koyfman A
      . The evaluation and management of urolithiasis in the ED: a review of the literature. Am J Emerg Med 2018;36:699–706.doi:10.1016/j.ajem.2018.01.003 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/29321112
      OpenUrlCrossRefPubMed
    9. ↵
      2. Fahey M ,
      3. Rudd A ,
      4. Béjot Y , et al
      . Development and validation of clinical prediction models for mortality, functional outcome and cognitive impairment after stroke: a study protocol. BMJ Open 2017;7:e014607.doi:10.1136/bmjopen-2016-014607 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/28821511
      OpenUrlAbstract/FREE Full Text
    10. ↵
      2. Pabinger I ,
      3. van Es N ,
      4. Heinze G , et al
      . A clinical prediction model for cancer-associated venous thromboembolism: a development and validation study in two independent prospective cohorts. Lancet Haematol 2018;5:e289–98.doi:10.1016/S2352-3026(18)30063-2 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/29885940
      OpenUrlPubMed
    11. ↵
      2. Eisner BH ,
      3. Goldfarb DS
      . A nomogram for the prediction of kidney stone recurrence. J Am Soc Nephrol 2014;25:2685–7.doi:10.1681/ASN.2014060631 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/25104802
      OpenUrlFREE Full Text
    12. ↵
      2. Okita K ,
      3. Hatakeyama S ,
      4. Imai A , et al
      . Stone episode prediction: development and validation of the prediction nomogram for urolithiasis. Int J Urol 2020;27:344–9.doi:10.1111/iju.14203 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/32147889
      OpenUrlPubMed
    13. ↵
      2. Rushton AB ,
      3. Verra ML ,
      4. Emms A , et al
      . Development and validation of two clinical prediction models to inform clinical decision-making for lumbar spinal fusion surgery for degenerative disorders and rehabilitation following surgery: protocol for a prospective observational study. BMJ Open 2018;8:e021078.doi:10.1136/bmjopen-2017-021078 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/29789351
      OpenUrlAbstract/FREE Full Text
    14. ↵
      2. Daines L ,
      3. McLean S ,
      4. Buelo A , et al
      . Clinical prediction models to support the diagnosis of asthma in primary care: a systematic review protocol. NPJ Prim Care Respir Med 2018;28:15.doi:10.1038/s41533-018-0086-6 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/29777106
      OpenUrlPubMed
    15. ↵
      2. Karel GM ,
      3. Moons M
      . New guideline for the reporting of studies developing, validating, or updating a multivariable clinical prediction model: the TRIPOD statement. Adv Anat Pathol 2015;22:303–5.doi:10.1097/PAP.0000000000000072
      OpenUrl
    16. ↵
      2. Steyerberg EW ,
      3. Vickers AJ ,
      4. Cook NR , et al
      . Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128–38.doi:10.1097/EDE.0b013e3181c30fb2 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/20010215
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      2. Moons KGM ,
      3. Wolff RF ,
      4. Riley RD , et al
      . PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med 2019;170:W1–33.doi:10.7326/M18-1377 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/30596876
      OpenUrlCrossRefPubMed
    18. ↵
      2. Ogundimu EO ,
      3. Altman DG ,
      4. Collins GS
      . Adequate sample size for developing prediction models is not simply related to events per variable. J Clin Epidemiol 2016;76:175–82.doi:10.1016/j.jclinepi.2016.02.031 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/26964707
      OpenUrlCrossRefPubMed
    19. ↵
      2. Zhang H ,
      3. Zhuang G ,
      4. Sun D , et al
      . Spontaneous rupture of the renal pelvis caused by upper urinary tract obstruction: a case report and review of the literature. Medicine 2017;96:e9190.doi:10.1097/MD.0000000000009190 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/29390332
      OpenUrlPubMed
    20. ↵
      2. Ganesan V ,
      3. De S ,
      4. Greene D , et al
      . Accuracy of ultrasonography for renal stone detection and size determination: is it good enough for management decisions? BJU Int 2017;119:464–9.doi:10.1111/bju.13605 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/27459091
      OpenUrlPubMed
    21. ↵
      2. Cormier CM ,
      3. Canzoneri BJ ,
      4. Lewis DF , et al
      . Urolithiasis in pregnancy: current diagnosis, treatment, and pregnancy complications. Obstet Gynecol Surv 2006;61:733–41.doi:10.1097/01.ogx.0000243773.05916.7a pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/17044950
      OpenUrlPubMed
    22. ↵
      2. Tibshirani R
      . The LASSO method for variable selection in the COX model. Stat Med 1997;16:385–95.doi:10.1002/(SICI)1097-0258(19970228)16:4<385::AID-SIM380>3.0.CO;2-3 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/9044528
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      2. Cooray SD ,
      3. Boyle JA ,
      4. Soldatos G , et al
      . Protocol for development and validation of a clinical prediction model for adverse pregnancy outcomes in women with gestational diabetes. BMJ Open 2020;10:e038845.doi:10.1136/bmjopen-2020-038845 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/33154055
      OpenUrlAbstract/FREE Full Text
    24. ↵
      2. Sanchez J ,
      3. Velasquez M ,
      4. Jaimes F
      . A protocol for the development and internal validation of a model to predict clinical response to antihistamines in urticaria patients. PLoS One 2020;15:e0239962.doi:10.1371/journal.pone.0239962 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/33022026
      OpenUrlPubMed
    25. ↵
      2. Stynes S ,
      3. Konstantinou K ,
      4. Ogollah R , et al
      . Clinical diagnostic model for sciatica developed in primary care patients with low back-related leg pain. PLoS One 2018;13:e0191852.doi:10.1371/journal.pone.0191852 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/29621243
      OpenUrlCrossRefPubMed
    26. ↵
      2. Türk H ,
      3. Ün S
      . Predictive factors for stone disease in patients with renal colic. Arch Ital Urol Androl 2017;89:143–5.doi:10.4081/aiua.2017.2.143 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/28679186
      OpenUrlPubMed
    27. ↵
      2. Moons KGM ,
      3. de Groot JAH ,
      4. Bouwmeester W , et al
      . Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the charms checklist. PLoS Med 2014;11:e1001744.doi:10.1371/journal.pmed.1001744 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/25314315
      OpenUrlCrossRefPubMed
    28. ↵
      2. van Smeden M ,
      3. Naaktgeboren CA ,
      4. Reitsma JB , et al
      . Latent class models in diagnostic studies when there is no reference standard--a systematic review. Am J Epidemiol 2014;179:423–31.doi:10.1093/aje/kwt286 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/24272278
      OpenUrlCrossRefPubMed
    29. ↵
      2. Wallace E ,
      3. Smith SM ,
      4. Perera-Salazar R , et al
      . Framework for the impact analysis and implementation of clinical prediction rules (CPRs). BMC Med Inform Decis Mak 2011;11:62.doi:10.1186/1472-6947-11-62 pmid:http://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/pubmed/21999201
      OpenUrlCrossRefPubMed

    Footnotes

    • Contributors Conceptualisation: LY, XZ and DX; funding acquisition: DK and LC; investigation and resources: LY and XZ; project administration, validation, visualisation, writing of the original draft, review and editing: LY; supervision: DX.

    • Funding This study is supported by Wenzhou Science and Technology Bureau Wenzhou, Zhejiang Province, China (project number: 20211021).

    • Competing interests None declared.

    • Patient and public involvement Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.