Article Text
PDF
Abstract
Objective The aim of this study is to develop, implement the precise reservation path (PRP) and investigate its prediction function for scheduling shunting patients for specialist appointment registration in Shanghai, China.
Design The PRP system was built on the hospital’s existing information system, integrated with WeChat (WeCom) for user convenience. The outcome analysis employed a mixed-methods approach, integrating quantitative analysis with statistical and machine learning techniques, including multivariate logistic regression, random forest (RF) and artificial neural network (ANN) analysis.
Setting This study was conducted at Renji Hospital, a premier general tertiary care institution in Shanghai, China, where the innovative PRP system was implemented. The programme was designed to efficiently connect patients requiring specialised care with the appropriate medical specialists.
Participants The PRP encompassed both voluntary specialists at Renji Hospital, as well as patients seeking outpatient specialist services.
Primary outcome measures The pass rates of patient for specialist applications.
Secondary outcome measures Clinical department, specialists’ and patients’ characteristics influencing specialist review result.
Results From a data set of 58 271 applicants across 26 departments between 1 December 2020 and 30 November 2022, we noted an overall pass rate of 34.8%. The departments of urology, breast surgery and thoracic surgery, along with five others, accounted for 86.65% of applications. Pass rates varied significantly, and demographic distributions of applicants across departments revealed distinct patient profiles, with preferences evident for age and gender. We developed an RF model based on pass rates from 26 specialised departments. The RF model, with 92.31% accuracy, identified age as the primary predictor of pass rates, underscoring its impact on specialist review outcomes. Focus on patient demographics, we conducted univariate and multivariate logistic regression analyses on the 58 271 patient data set to explore the relationship between demographic factors and review outcomes. Key findings from logistic regression included significant associations with gender, age and specialist title. Results indicated that older patients were more likely to be approved in specialist reviews, while middle-aged patients had lower pass rates. The generalised linear model, enhanced with specialist and clinical department variables, showed superior predictive accuracy (67.86–68.26%) and model fit over the previous logistic model. An ANN model also identified specialist and clinical department as the most influential, achieving comparable accuracy (67.72–68.28%).
Conclusions The PRP programme demonstrates the potential of digital innovation in enhancing the hierarchical medical system. The study’s findings also underscore the value of the PRP programme in healthcare systems for optimising resource allocation, particularly for ageing populations. The programme’s design and implementation offer a scalable model for other healthcare institutions seeking to enhance their appointment systems and specialist engagement through digital innovation.
- eHealth
- Hospitals
- Patients
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The study takes a comprehensive approach by integrating perspectives from both departments and patients, leading to a more complete analysis than if only one viewpoint were considered.
The research employs a combination of traditional statistical methods (such as logistic regression), machine learning techniques (like random forest) and artificial neural networks to analyse the data, offering a well-rounded understanding of the elements influencing specialist appointment outcomes.
There was a possibility of personal bias in expert assessments, which could affect the precision of the specialist review findings.
Financial constraints limited the capacity to conduct a more detailed analysis of patient-disease condition descriptions, which could be crucial for improving predictive models.
Introduction
Most people seek affordable, dependable and high-quality products and services such as electronics, auto repairs and massage. This is also true of the healthcare sector. High-quality healthcare is defined as care, that is, effective, safe, patient-centred, timely, efficient, equitable and delivered by professionals who communicate respectfully, communicate clearly and involve patients in decision-making.1 To achieve the goals of ‘Healthy China 2030’, China has deepened health reforms in recent years to establish high-quality, value-based service delivery, resulting in a series of significant improvements. However, several challenges persist, including the uneven distribution of healthcare resources and the imbalance between supply and demand.2 3 Improving the efficiency of hospital care is, to the best of our knowledge, a fundamental aspect of strengthening the health system. In China, public hospitals are the main body of the medical service system and the primary places for people to seek medical treatment. According to the China Health Statistics Yearbook, in 2018, public hospitals provided 92% of outpatient services and 82% of the inpatient services among all hospitals in China. According to China’s health delivery structure, public hospitals are organised into a three-tier system (designated as primary, secondary or tertiary institutions). A tertiary hospital is a comprehensive, referral and general hospital at the city, provincial or national level with a bed capacity exceeding 500. Tertiary hospitals are responsible for providing specialist health services, performing a more significant role in medical education and scientific research and serving as medical hubs that offer care in multiple regions.4 Since China’s healthcare system does not feature a gatekeeping general practitioner (GP) system, patients can seek primary care from primary care facilities or hospital outpatient departments.5 Considering the free choice of healthcare providers, most patients prefer to select a specialist (who is a medical doctor, often with a senior professional title and an expert in a specific area of medicine) in clinics, private hospitals or public hospitals, particularly tertiary public hospitals. However, due to unequal access to information between doctors and patients, the freedom to choose healthcare providers often leads patients to select doctors without adequate knowledge or understanding, potentially resulting in a significant misallocation of medical resources.6 This is because patients may lack the necessary information to determine the most suitable doctor for their needs. Additionally, patients with minor illnesses preferred to visit tertiary hospitals instead of primary care centres. These behaviours lead to the substantial misuse of valuable medical resources, creating a dual challenge. First, it imposes a heavy economic burden on patients, potentially exceeding the affordability limits of the medical system. Second, these behaviours pose obstacles to the high-quality development of tertiary hospitals in China. Moreover, the scarcity of healthcare resources has led to fierce competition among patients.7 8 To support the provision of high-quality and timely outpatient services, many hospitals have implemented innovative appointment registration systems to assist patients and increase hospital efficiency. These systems allow hospitals to provide services more efficiently. Under the rule of ‘first registration, first service’ in appointment registration systems, some patients with severe diseases requiring specialist treatment may struggle to book appointments with the necessary specialists successfully. This inequity in healthcare resource allocation contradicts the principles of China’s current medical reform. Thus, it is imperative to optimise the appointment registration system and improve the efficiency of medical procedures in Chinese tertiary public hospitals. Accordingly, hospitals are testing novel appointment registration systems, such as mobile phones, web-based systems, bank hospital partnerships and clinical settings. Among these, redesigning the appointment-scheduling system is crucial for effectively using healthcare resources, increasing operational efficiency, reducing operational costs and mitigating the imbalance between supply and demand for healthcare services.9
Over the past decade, China has had the highest number of smartphones per capita among all the countries.2 Equipped with ‘Internet Plus Medical’ applications, online appointment systems have become an alternative that optimises scheduling processes, significantly improving the efficiency of hospital services. Hospitals provide various web-based services through mobile platforms, including WeChat and other independently developed applications. Hospital services include online consultations, appointment registration and payments. Such mobile health initiatives can overcome geographical boundaries, enhance the equity and accessibility of healthcare resources and provide practical and equitable access to healthcare services in hospitals.10 Mobile appointment registration services provide patients with an essential online channel through which those with smart devices can access healthcare resources. With the availability of mobile appointment registration services, patients will increasingly turn to online platforms for booking appointments, gradually replacing traditional offline queuing registrations. This shift to online registration is expected to create a higher demand for services on mobile health platforms. In hospitals, the outpatient department plays a critical role in efficiently managing medical resources, as it not only directs patients toward timely care but also generates numerous benefits, including improved health outcomes and optimised utilisation of healthcare resources. As discussed, optimising appointment registration procedures is the responsibility and commitment of hospital management. Thus, the effective scheduling of mobile appointment services and the provision of healthcare services to patients have become critical priorities for outpatient departments in tertiary public hospitals in China.
This study aims to develop a precise reservation path (PRP) framework to ensure timely and efficient treatment for patients with severe diseases in tertiary public hospitals in Shanghai, China. This study first describes a specialist-led reservation process in which the clinical data (history, examination and past treatment) of a reservation applicant are assessed using relevant ultrasound, laboratory or radiological records. Subsequently, a decision is made regarding the suitability of the patient for specialist intervention. Second, by employing specific statistical analysis methods, our objective is to investigate the factors affecting the pass rate of patients who require specialist medical services, thereby streamlining their referral to appropriate healthcare settings, such as specialised clinics or primary care institutions. This system aims to strengthen and improve the hierarchical diagnostic and treatment systems in China.
Materials and methods
Patient and public involvement
This study did not involve direct patient or public participation.
Design
Procedure of PRP
This study was conducted at the Renji Hospital, School of Medicine, Shanghai Jiao Tong University, a 2750-bed general tertiary public hospital in Shanghai, China. To fully digitise its records, the hospital has established an information infrastructure in which most medical and health records are stored electronically. The annual number of outpatient and emergency visits to hospitals exceeds 5.82 million, and the annual number of discharged patients is 172 000. The PRP programme was developed by the management and clinical staff at Renji Hospital in Shanghai to address the disorganised system in which outpatients schedule specialists without restrictions. Before designing the PRP system, a literature search, peer discussions, panel discussions and specialist consultations were conducted to determine its structure and design. Staff from the outpatient and emergency management departments, as well as specialised surgeons and information technology technicians in thoracic surgery, breast surgery and urinary surgery, attended meetings to design the PRP process. The PRP process was designed as follows: Our main objective was to build a cloud-based, precise medical reservation path connecting all participants (patients and specialists). Based on the PRP framework, patients with suspected severe disease can request a specialist appointment and submit their medical records. After a specialist reviews the patient’s application and medical records, the specialist may approve or reject the patient’s application. Given that patients often directly schedule outpatient appointments at tertiary hospitals without undergoing primary healthcare screening, there is a free choice of healthcare providers. In the Chinese healthcare system, 935 625 specialist outpatient visits were identified as not progressing through the system during the study period, from 1 December 2020 to 30 November 2022, at Renji Hospital. The PRP flowchart is shown in figure 1.
The precise reservation path flow chart.
Architecture of PRP
The architecture of the PRP model is shown in online supplemental figure S1. Technical support ensured that different system modules were compatible and established a cloud-based PRP system that connected it to the original hospital information system (HIS) of Renji Hospital. Building on the original HIS, new modules and functions were added to share all specialist outpatient schedules and specialist team accounts, such as WeChat (WeCom). The PRP system was integrated with hospital scheduling systems, enabling the automatic identification of appointment statuses, including whether an appointment had been made, requested or cancelled (and changing the status if rescheduling is required). The specialist team user has one-click access to review the patient’s application, examine the notes in the submitted medical records and communicate with the team regarding the treatment plans, thereby determining the need for scheduling further evaluation.
Supplemental material
Operation of PRP
The PRP is based on the official WeChat account platform of the hospital
WeChat (WeCom) is a free social networking application that offers instant messaging services across all platforms. It provides basic text, voice, photos, video sharing, web-based payments and integration with intelligent hardware. A financial report from Tencent indicates that the number of combined monthly live WeChat accounts exceeded 1.2 billion by the end of March 2021. Using the WeChat framework, we developed a function that is compatible with both iOS and Android platforms. Therefore, our design had a minimum learning cost for the participants and ensured the sustainability of users.
PRP is based on an asynchronous form
Online information exchange between patients and doctors can occur synchronously (when interactions arise in real time) or asynchronously (with a delay between transmission and response).11 12 We adopted an asynchronous form of PRP because doctors in hospitals are always busy and cannot guarantee that they are available online in real time. Additionally, the asynchronous format maximises doctors’ fragmented time to process and respond to patient applications.
PRP is also based on cloud computing services
Cloud computing provides on-demand access to computer system resources, particularly data storage (cloud storage) and computing power, without requiring active user management. It offers network access to a shared pool of configurable computing resources as a metered, on-demand service that efficiently distributes resources. The cloud-computing architecture offers several benefits, such as cost reduction, device and location independence, easier maintenance and on-demand self-service. Similarly, an essential feature of mobile health services is on-demand self-service, where users can instantly access network storage and server processing. Therefore, the PRP was designed using cloud computing services.
PRP involving users of PRP
PRP consists of two types of users: patients and specialists. The Mobile Outpatient Specialist Appointment Registration Procedures are presented in online supplemental table S1.
Investigation of the PRP programme
Data resources and measures
Renji Hospital began implementing PRP in December 2020. Data used in the study were retrieved from the official WeChat medical service account and HIS at Renji Hospital from 1 December 2020 to 30 November 2022. In total, 58 271 samples were obtained.
The specialist review outcomes were dichotomised as either unpassed (=0) or passed (=1) as the dependent variable. The independent variables included sex, age, preference for paying insurance and the title of the specialist. The classifications and measures of the variables of interest are listed in online supplemental table S2.
Analysis strategy
Traditional statistical method and multivariate logistic regression method
First, all continuous variables were tested for normality. The Kolmogorov-Smirnov test results (Sig=0.070>0.05) showed that ‘age’ was normally distributed in the 58 271 patients. Means±SD were used for variables that had a normal distribution, and median and IQRs were reported for non-normally distributed data. 85 specialists from 26 specialties participated in the PRP programme from 1 December 2020 to 30 November 2022. Univariate analysis was conducted to identify significant contributors to the dependent variable, that is, whether the patient passed the online specialist appointment registration confirmation. In this study, the χ2 test was used for univariate analysis to assess categorical variables. Finally, multivariate logistic regression, with the backward stepwise method, was performed to identify the main factors influencing the specialists’ review outcomes.
Random forest analysis: based on participating departments
We collected the operational status of each department in the PRP project and the allocation of specialist resources, which were designated as input variables. The pass rates for the PRP among departments were defined as the target variable. Due to its everyday use in the random forest (RF) algorithm machine learning (ML), RF can be applied to handle binary and multiclassification problems or examine interaction variables. Additionally, it can provide variable importance measures for categorical and continuous variables, potentially involving those with high noise and significance. RF also demonstrates strong predictive performance, even for data with more variables (p) than samples (ie, p>n). Due to their non-parametric nature, RFs are a robust method used in relatively straightforward applications for inexperienced users. Previous research included the RF analysis method for small sample sizes, even fewer than 20 (denoted as N<20). Therefore, to assess the relative significance of these input variables in predicting the target, we performed an RF algorithm analysis. Since training and testing an ML model on the same data set can lead to overfitting—when a model captures noise and random fluctuations instead of underlying patterns—an ‘out-of-bag’ (OOB) technique was applied to make predictions using only base learners not trained on each particular observation. This method is known as the OOB prediction. These predictions are not prone to overfitting because each prediction is made only by learners who do not use observations for training. Due to the presence of OOB samples in our data set, the OOB data can be used directly as a validation and test set, eliminating the need to prepare the data set. In the SPSS Modeler, the predictive accuracy is precisely the result of OOB estimation.13–18
Generalised linear model
Generalised linear model (GLM) identifies the dependent variable as linearly related to the factors and covariates via a specified link function. Moreover, the model allows the dependent variable to have a non-normal distribution. It covers widely used statistical models such as linear regression for normally distributed responses, logistic models for binary data, log-linear models for count data, complementary log–log models for interval-censored survival data and many other statistical models through its general model formulation.
Artificial neural network
Artificial neural networks (ANNs) are mathematical models that are based on interconnected groups of artificial neurons. They considered non-linear relationships between the input data, which are not always identified in traditional analyses. ANNs have several advantages, including self-learning, adaptability and robustness due to massive parallelism. They generally consist of three layers: input, hidden and output layers. A multilayer perceptron (MLP) is a subtype of an ANN comprising one or more hidden layers with computation nodes capable of universal approximations. Therefore, it has been extensively used for modelling non-linear and complex processes as well as real-world processes. The most widely used algorithm for training an MLP network is the back-propagation (BP) algorithm.19–23 The data set was randomly divided into training and test sets in a 70:30 ratio (40 766 and 17 505 records, respectively). All statistical analyses were performed using SPSS software (V.23.0; IBM, Armonk, New York, USA). IBM SPSS Modeler V.18.0 was used to construct the RF, GLM and BP-ANN models.
Results
Result of operation status of each department in the PRP programme
From a data set encompassing 58 271 applicants, we observed an overall passing rate of 34.8%. We collected data from 26 departments, including the number of applications, pass rate, median age of patients, IQR of patient ages, sex ratio, proportion of medical insurance patients, patient application count for senior specialists, patient application count for associate senior specialists, senior specialist applicant ratio, PRP programme specialists, total specialists, PRP specialists and average specialist applications. As shown in online supplemental table S3, the operational status of each department in the PRP programme can be comprehensively and visually understood. The proportion of senior specialist applications varied significantly across departments, indicating different levels of expert engagement in the PRP programme. The average number of applications per specialist also reflects the popularity of the PRP programmes within each department. The eight departments with the highest number of patient applications were urinary surgery (11 142), breast surgery (10 097), thoracic surgery (9332), obstetrics and gynaecology (4862), gastrointestinal surgery (4716), nephrology (4011), head and neck surgery (3256) and biliary and pancreatic surgery (3076). The total number of applications for these departments accounted for approximately 86.7% of all applications, highlighting the importance of their importance in the PRP programme and the significant demand from patients for their services.
Among these departments, those with high pass rates included head and neck surgery (52.7%), biliary and pancreatic surgery (47.3%), obstetrics and gynaecology (44.8%), nephrology (42%), gastrointestinal surgery (38.6%) and urinary surgery (37.4%). Breast surgery had the lowest pass rate (18.3%). In terms of age distribution within these departments, urinary surgery had the highest median patient age (64 years, IQR=17), primarily serving middle-aged and elderly patients; breast surgery (44 years, IQR=20) caters to a relatively younger patient population with a more dispersed age distribution; and thoracic surgery (56 years, IQR=22) and gastrointestinal surgery (59 years, IQR=26) have a broader age range, covering patients from middle age to the elderly. The sex ratio data reflected the characteristics of sex distribution among patients served by different departments. In terms of age distribution within these departments, urinary surgery had the highest median patient age (64 years, IQR=17), primarily serving middle-aged and elderly patients; breast surgery (44 years, IQR=20) caters to a relatively younger patient population with a more dispersed age distribution; and thoracic surgery (56 years, IQR 22) and gastrointestinal surgery (59 years, IQR 26) have a broader age range, covering patients from middle age to the elderly. The sex ratio data reflected the characteristics of sex distribution among patients served by different departments. For instance, urinary surgery and breast surgery show extreme sex ratios, indicating clear male and female preferences, respectively. In contrast, nephrology and biliary and pancreatic surgeries have more balanced sex ratios.
Result of RF analysis based on 26 department
Based on the pass rates from 26 specialised clinical departments, we used IBM SPSS Modeler V.18.0 to develop an RF model. Under the guidance of a statistical expert, model parameters were calibrated, with the number of models to build set to seven and the sample size meticulously configured to 1. To enhance the efficacy and feature selection of the model, we activated the ‘Handle imbalanced data’ feature and the ‘Use weighted sampling for variable selection’ option. For tree growth parameters, we retained the default settings provided by the software, which included a maximum number of nodes set to 10 000, a maximum tree depth of 10 and a minimum child node size of 5.
In this study, the overall pass rate was 34.8%. We have set the ‘Pass Rate Interval’ field to categorise departments as either ‘high-rank group’ or ‘low-rank group’ based on this rate. Based on this, the ‘Pass Rate Interval’, a binary variable, is taken as the target variable in the RF analysis. The variables, including patient age median, PRP specialist proportion, average specialist applications, total specialists, insurance patient ratio, PRP programme specialists, patient sex ratio, application count and senior specialist applicant ratio, all served as input features in the RF analysis. The accuracy of the model was 92.31%. The predictor importance values range from 0.12 to 0.93, indicating the relative significance of each variable in predicting the target variable. Median patient age emerged as the most significant predictor with a value of 0.93, followed by PRP specialist proportion and average specialist applications with values of 0.87 and 0.84, respectively. However, variables such as application count and senior specialist applicant ratio had lower importance values, indicating their smaller contribution to the prediction process. Online supplemental table S4 and figure S2 present an overview of the variables and their respective roles in an RF model, along with their predictor importance.
Overview and basic characteristics of patients
85 specialists from 26 specialties participated in the PRP programme from 1 December 2020 to 30 November 2022. During this period, 58 271 patients applied for specialist appointments through the PRP programme. In China, patients typically schedule outpatient appointments at tertiary hospitals without prior approval. Of these 58 271 patients, 20 290 (34.8%) passed the specialist assessment and 37 981 (65.2%) did not.
Of the 58 271 patients, the mean age was 52.89±15.73 years. The Kolmogorov-Smirnov method test results (Sig=0.070>0.05) showed that ‘age’ was normally distributed in the 58 271 patients. Among them, 22 412 (38.5%) were male, and 35 859 (61.5%) were female. See table 1 for more details on patient characteristics.
Patient characteristics, univariate and multivariate logistic regression analysis of factors related to the specialist’s review outcome
Results of univariate and multivariate logistic regression analysis in the total population sample (N=58 271)
Focusing on patient demographics, we conducted univariate and multivariate logistic regression analyses of a data set of 58 271 patients to explore relationships between demographic factors and assessment outcomes (table 1). Univariate analyses revealed that the significant factors associated with specialist review outcomes were sex (p<0.001), age (p<0.001) and specialist title (p<0.001). Age, sex and specialist title were found to be significant in the univariate analyses and were subsequently included in the multivariate logistic regression model. This model was constructed using the backward stepwise method to refine variable selection. Then, through a multivariate logistic regression model, it indicates that those who were male and aged 60 years old or above were more likely to pass the specialist’s review in the PRP programme. At the same time, those aged 35–59 years were less likely to pass the specialist’s review in the likely PRP programme compared with those aged 18–34 years old. Additionally, those who had the title associate senior were more likely to pass the specialist’s review than those who had the title of senior specialist (see table 1).
Nagelkerke’s R2 ranged from 0 to 1, with values closer to 1 indicating a better model fit. However, as shown in table 1, the value of Nagelkerke’s R2 was 0.016. From this, we speculate that more ‘strong’ variables are not included in the predictive model of the expert review results (table 1). Therefore, we added a GLM to identify additional predictors not included in the multivariate logistic regression model to enhance model fit.
Result of GLM analysis
In this enhanced analysis, we incorporated two additional variables, specialist and clinic, into the independent variables of the GLM and ANN models. The target variable was the assessment result, which was categorised as a dummy variable. The GLM regression results show the specialist and clinic as the most significant variables, with a relative importance of 0.71 and 0.26, respectively. In contrast, age and specialist title exerted a comparatively minor influence, each with a significance of 0.02 (see table 2). Furthermore, the predictive accuracy of the model is outlined in table 3, with scores of 67.86% and 68.26% for the training and test data sets, respectively. The Akaike information criterion (AIC) value for the GLM, 48 433.067, was significantly lower than the AIC value of 52 335.946 for the multivariate logistic model. This decrease in the AIC indicates that the GLM, with its five input variables, outperforms the multivariate logistic model, which has only three inputs in terms of model fit and explanatory power.
The role in GLM and ANN model and result of predictor importance
Accuracy of GLM and ANN analyses
Result of ANN analysis
To further understand how patient characteristics and specialist choices affect PRP assessment results, we developed an ANN model with an MLP structure. We maintained all other parameters at their default values as specified by the software. In the ANN analysis shown in table 2, the input features encompass specialist, clinical department, age, title of the specialist and sex, with assessment result (yes or no) serving as the target variable. Table 2 further delineates the key variables influencing the assessment result within the ANN model, ranking specialist and clinical department as the most impactful, with the relative importance of 0.44 and 0.43, respectively. Age, specialist title and sex had less influence at 0.07, 0.04 and 0.02, respectively. Figure 2 illustrates the topological architecture of the ANN model. The accuracy of the model is shown in table 3, with scores of 67.72% and 68.28% for the training and test data sets, respectively. Online supplemental table S5 presents an overview of the variables and their respective roles in an ANN model, along with their predictor importance.
Topology architecture of the artificial neural network model.
Discussion
Digital technologies increasingly support health systems (WHO, 2018) by providing flexible options for interpersonal communication and information exchange.24 Access, affordability and equity are the three primary goals of a well-functioning health system.25 In this study, among the 58 271 patients who applied to medical specialists on the PRP-based platform, 20 290 successfully passed reviews conducted by specialists, thereby gaining access to specialised medical treatment in hospitals. This indicates that the mobile health-based PRP programme is a practical and innovative approach. This enables patients to obtain convenient access to medical specialists in hospitals, with reduced doctor-seeking costs, thereby ensuring timely and highly efficient treatment. The remaining 37 891 patients who did not pass the specialist review were identified through the PRP programme. Specialists advised them to be appropriately directed to other clinical departments within the hospital or primary care centres for treatment by GPs. This guidance helps prevent inefficiency by discouraging patients from blindly seeking specialist appointments, which could otherwise lead to a waste of valuable medical resources in hospitals.
Implemented across 26 departments, the PRP programme demonstrated diverse performance, with significant variations in operational metrics such as application volumes, pass rates and specialist involvement. The implemented programme attracted 58 271 patients who applied for specialist appointments during the study period, reflecting the extensive reach and the significant demand of the programme for specialist services from outpatients. The smooth implementation of the programme was highlighted by the high participation rate of 85 specialists from various fields, demonstrating a robust network of professionals eager to engage in the PRP initiative.
Univariate analysis revealed that age was significantly associated with the outcomes of specialist reviews (p<0.001). Logistic regression showed that individuals aged 60 years and above were more likely to pass reviews compared with the 18–34 age group, whereas the 35–59 age group was less likely to do so. The RF analysis identified applicant age as the most important predictor, with an importance value of 0.93, in explaining differences in pass rates among the 26 clinical departments. This may be because the elderly, who often suffer a variety of chronic diseases, may have higher medical needs than younger individuals.26 This is consistent with the evidence that many previous studies have claimed that elderly patients have more significant utilisation of hospital outpatient or inpatient healthcare services than their younger counterparts because of their unfavourable health status.27 Conversely, those aged 35–59 had a lower probability of passing the specialist review and a higher likelihood of being genuinely rejected than those aged 18–34. This could be explained by the fact that middle-aged patients prefer advanced medical care regardless of their medical needs. These patients may see specialists in hospitals for psychological comfort or because they mistrust GPs in primary care settings even though they do not require special medical care. Many previous studies have reported that this phenomenon results from the lack of community gatekeeper systems in China.28 29 This reflects the urgent need to strengthen primary care and family doctor services for middle-aged people in China. Additionally, male patients were more likely to pass the review than their female counterparts were. This may be explained by the higher prevalence of chronic diseases among males than among females in China.30 However, this finding was not consistent with previous studies, all of which reported a higher prevalence of chronic diseases in females, especially in those aged 60 years and above in China.31 32 However, the sex ratios in the prevalence of chronic diseases remain inconclusive. Therefore, the Shanghai government and decision-makers in the health sector should make gender-specific and age-specific efforts to facilitate the implementation of hierarchical diagnosis and treatment systems and family doctor services for middle-aged patients. By shunting middle-aged patients to GPs or secondary hospitals, patient crowding caused by the ‘siphon effect’ can be mitigated, thus improving the efficiency of procedures in tertiary hospitals.
More interestingly, specialists who held senior associate titles were more likely to approve patients’ applications than senior specialists were. This may be due to the intense competition for title promotion in medicine. In Chinese society, job hierarchy plays a significant role, which is evident in the use of professional titles.33 34 Professional titles often signify seniority, experience or level of authority within government-funded organisations. Given the pressure for promotion, specialists with associate senior titles prefer to accept more patients to enhance their professional skills and prepare for promotion to senior professional titles. The analyses of the GLM and ANN models reveal a distinct prioritisation of specialist and departmental factors within the PRP programme, surpassing the predictive power of patient age and gender, which may stem from a complex array of reasons as follows: First, the unique expertise of specialists is of paramount importance; for instance, urologists may specialise in different subspecialties, with some focusing on kidney conditions, others on prostate issues, and still others on bladder diseases, all of which substantially shape patient requests and the outcomes of assessments.
Second, specific clinical departments have a more concentrated age range among their applicants, with notably skewed gender demographics. For instance, the breast surgery department is characterised by a significant prevalence of middle-aged women, with a median age of 44 years and a sex ratio of 0.01 among its applicants. In contrast, the urology department has a predominantly male applicant base, with a sex ratio of 3.85, and applicants are mostly older, with a median age of 68 years, which exceeds the overall median age of the PRP applicant pool by 14 years. Together, these two departments constitute a considerable portion of the PRP applicant pool (36.45%). The unique demographic characteristics of these specific departments may profoundly influence predictive factors.
This study has several significant limitations. First, the potential for personal bias in expert evaluations may be considerable. In the future, we plan to incorporate rule-based systems with multimodal artificial intelligence assessments to reduce the impact of individual biases. The GLM model achieved a score of 67.86% on the training set and 68.26% on the test set, whereas the ANN model achieved a score of 67.72% on the training set and 68.28% on the test set, showing a nearly equivalent predictive performance. The accuracy of the GLM and ANN models on the testing data set suggests a significant opportunity to improve and enhance their predictive accuracy. Second, existing budget limitations hinder the ability to conduct a structured analysis of patient-disease condition descriptions, highlighting the need for further research. A forward-looking approach is essential for creating a more robust, comprehensive and inclusive predictive model by incorporating this crucial variable.
Conclusion
The PRP programme offers a promising approach to integrating digital health technologies into healthcare systems. It provides a scalable and adaptable model for optimising specialist appointment processes. Our future goal is to enhance and expand the programme by involving more specialists and hospitals, thereby including a larger number of patients. Healthcare entities seeking to improve appointment processes and specialist engagement through digital advancements may refer to the design and implementation of the programme.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
Ethical approval was obtained from the Human Research Ethics Committee of Renji Hospital (LY2023-031-B). Written informed consent was not required in accordance with the ethics approval.
Acknowledgments
We would like to thank Department of Information Technology of Renji Hospital for their assistance and support in the field research.
Footnotes
Contributors MC, BZ, YF, TZ and ED contributed to the conception and design. MC, YF, TZ and ED contributed to statistical analyses. MC, Xiaojing Zhao, Xuji Zhao, WS and LL contributed to data acquisition and data interpretation. MC and ED drafted the article. YF is the guarantor. All authors revised the manuscript and approved the final version of the manuscript.
Funding This study was sponsored by The National Social Science Foundation of China General Project (Grant No. 18BGL242; 19BGL246); National Social Science Foundation of China Major Project (Grant No. 18ZDA088); high-level local university cultivation projects of Shanghai University of Medicine and Health Sciences (E1-2601-22-201006-3). The sponsors were not involved in the design and conduct of the study; the collection, management, analysis and interpretation of data; or the preparation, review and approval of the manuscript.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.