You are here

Article Text

Article menu

Download PDFPDF

General practice / Family practice
Original research
Patient-facing online triage tools and clinician decision-making: a systematic review
  1. Armina Paule1,
  2. Gary A Abel2,
  3. Jo Parsons3,
  4. Helen Atherton4
  1. 1Applied Health, Warwick Medical School, University of Warwick, Coventry, UK
  2. 2Department of Health and Community Science, University of Exeter, Exeter, UK
  3. 3Birmingham Centre for Evidence and Implementation Science (BCEIS), University of Birmingham, Birmingham, UK
  4. 4Primary Care Research Centre, University of Southampton, Southampton, UK
  1. Correspondence to Armina Paule; armina.paule{at}warwick.ac.uk

Abstract

Objective To evaluate the role of using outputs from patient-facing online triage tools in clinical decision-making in primary care.

Design Systematic review.

Data sources Medline, Embase, Cumulative Index to Nursing and Allied Health Literature, Web of Science and Scopus were searched for literature published between 1 January 2002 and 31 December 2022 and updated for literature published up to end of November 2024.

Eligibility criteria for selecting studies Studies of any design are included where the study investigates how primary care clinicians make clinical decisions in response to patient concerns reported using online triage tools.

Data extraction and synthesis Data were extracted, and quality assessment was conducted using the Mixed Methods Appraisal Tool. Narrative synthesis was used to analyse the findings.

Results 14 studies were included, which were conducted in the UK (n=9), Sweden (n=3) and Spain (n=2). There were no studies that examined clinical decision-making as an outcome. Outcomes relating to the impact on clinical decision-making were grouped into three categories: patient clinical outcomes (n=9), primary care practitioner experience (n=11) and healthcare system outcomes (n=14). Studies reported faster clinical decisions made in response to patient concerns. Other studies reported clinicians offering unnecessary urgent appointments as patients learnt to ‘game’ the system. Clinicians felt confident managing patient requests as they can access additional information (such as a photo attachment). Moreover, clinicians’ time was freed up from appointments with limited clinical value. Contrarily, online triage was perceived as an additional step in the workflow.

Conclusion Clinicians should be aware that their decision-making processes are likely to differ when using online triage tools. Developers can use the findings to improve the usability of the tools to aid clinical decision-making. Future research should focus on patient-facing online triage tools in general practice and the process of clinical decision-making.

PROSPERO registration number CRD42022373944.

  • Triage
  • Primary Care
  • Primary Health Care
  • Digital Technology

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Data availability statement

Relevant data are included in online supplemental tables.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi-org.ezproxy.u-pec.fr/10.1136/bmjopen-2024-094068

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    STRENGTHS AND LIMITATIONS OF THIS STUDY

    • This is the first systematic review to focus on outcomes relating to clinical decision-making from triaging patients who completed an online form to contact their primary care service.

    • This review compares the tool usage across different countries with universal healthcare coverage, enabling insights from early adopters of the technology.

    • This review covers a 20-year period to enable the evaluation of older literature. However, the number of included studies was very small, because this topic is not sufficiently studied yet.

    • The review was limited to studies published in English, which may have led to some evidence being overlooked.

    Background

    Patient-facing online triage tools in primary care facilitate contact between the patient and their primary care service provider. The tools gather information about patients’ clinical needs to enable triage decisions on the appropriate next steps. A triage decision is usually focussed on how best to meet the identified patient healthcare needs. Patient’s needs might be resolved over a telephone consultation, by issuing a repeat prescription, requiring the patient to be seen by a healthcare professional (eg, a general practitioner (GP) or a nurse) or requiring escalation to secondary care.1 In short, triage is a prioritisation process that has long been proposed as a solution to management of the demand for primary care services.2–5

    Both NHS England (the National Health Service Commissioning Board for England)6 and the Royal College of General Practitioners7 have taken an interest in encouraging the use of patient-facing online triage tools in primary care. Online triage is part of the digital transformation that ensures better patient access to primary care.8 Online triage tools allow the patient, their carer or non-clinical reception staff at the general practice to complete a form outlining the reason for contacting their primary care general practice. Clinicians then review the submitted form and use it to prioritise patients based on their clinical needs. There are reported challenges facing the adoption of remote triage across Northern Europe.9 10 These challenges include primary care clinicians’ workload, in particular whether the tools create additional work or reduce workload.11 12 Some online tools are now using artificial intelligence (AI) (whether rules based or using machine learning) to enable these tools to flag key complaints and simplify the reviewing of forms by the clinician.13

    Clinical decision-making is the process where a clinician combines the information reported by the patient with their expert judgement to make the best decision on the patient’s clinical journey.14 The clinical decision-making process using reports from online triage tools is influenced only by information reported on the triage form and lacks visual and verbal cues (eg, eye contact, patient voice and the patient context) present when patients consult in person.15 16 Clinicians are often guided by ‘gut feelings’ which might only occur if the triage clinical decision leads to a telephone or an in-person follow-up rather than a written response.17 However, outcomes like safety netting18 19 (eg, when a doctor asks the patients to book another consultation if their symptoms do not improve in a few weeks) are possible in online triage.

    Previous systematic reviews have examined this topic with a focus20 21 on examining patient safety, timeliness, efficiency, equitability and patient-centredness as outcomes of using online communication tools in primary care. Darley et al’s20 review included a wide range of digital and online tools and outcomes and analysed the literature from the perspectives of patients, clinicians, tool developers and policymakers without analysis of the impact on clinical decision-making. Additionally, the review focused on implementation at practice level, not clinical application of patient-facing online triage tools. The other systematic review by Mold et al21 focused on tools where patients respond to their primary care provider using secure email and messaging or video links in primary care. Mold et al’s review differs from this systematic review in that the patient-facing online triage tools examined in this study require the patients to initiate the contact with their primary care provider.

    This review examines how primary care clinicians make clinical decisions when using outputs/reports from patient-facing online triage tools, given that they have become more widely adopted by general practices in recent years.

    Methods

    This systematic review examines how primary care clinicians use outputs from patient-facing online triage tools for clinical decision-making in primary care and the associated clinical, patient and health system outcomes.

    This systematic review is reported according to the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) framework22 and following the completed PRISMA checklist attached in online supplemental appendix 1.

    Patient and public involvement

    No patient and public involvement directly fed into the development or conduct of this review.

    Eligibility criteria

    The population, interventions, comparators, outcomes and study design principles were used to develop the eligibility criteria.

    Population: Primary care clinicians using output or reports from patient-facing online triage tools to make a clinical triage decision, where the patients completed the triage requests and were adult patients.

    Interventions: Studies that assessed web or app-based tools used in primary care setting where the patient, their carer or general practice receptionist completes a form for a clinician to review and triage. Including but not limited to tools that are using AI.

    Studies were excluded where the intervention tools being assessed met at least one of the following criteria: being used only outside of primary care settings, being used to assess specific symptoms and features rather than providing triage (eg, symptoms of diabetes and body mass index (BMI) check), being digital symptom checker platforms, providing likely diagnosis, not informing triage, providing access to a direct primary care consultation or still being under development.

    Comparators: Studies were not required to have a comparator, and any comparator group was considered.

    Outcomes: Studies that assessed clinical decision-making as an outcome or studies that assessed at least one outcome related to clinical decision-making. The outcomes related to clinical decision-making are defined as any outcome that may be impacted by a change in the quantity and quality of information available to the clinician, due to a change of information source (in-person vs patient-facing online triage tool) when making a clinical decision, if everything else remains the same (tools to process the information and the clinician level of expertise). Therefore, the following outcomes relating to clinical decision-making were used as a guide: (1) clinical outcomes such as diagnosis, severity of diagnosis, time to treatment, time to first investigation, time to referral, alignment with professional guidance on investigation, treatment or referral; (2) primary care practitioner experience such as confidence in diagnosis and comfort with decision-making; (3) healthcare system outcomes such as frequency of primary care appointments via different modes, clinical workload, number of Emergency Department attendances and emergency admissions. Outcomes relating to clinical decision-making were included whether assessed at an individual clinician level or system level (eg, general practice). Studies that examined hypothetical clinician experience and that focus on patient satisfaction of using the tools were excluded. The outcomes outlined are a guide to what is expected to find in search results and allow the authors to group the data. This ensures that any outcome that is relevant to the question will be included. The outcomes were selected based on input from subject matter experts (BH and JD) in primary care.

    Study design: All empirical study designs were eligible for inclusion: qualitative (case studies, interviews, focus groups, observational notes, open-ended surveys), quantitative (cohort studies, case reports, secondary data analysis) and mixed methods studies. Systematic reviews were excluded.

    Search: There was no restriction on the country of study, and only studies published in English were eligible for inclusion. The decision to include non-UK-based publications was taken to identify the similarities and differences in outcomes from different countries in relation to the diverse models of primary care service delivery, for example, the variation in the role of GPs and patients’ access to healthcare services in different countries.

    The initial search was limited to articles published between 2002 and 2022 reflecting that the tools of interest emerged over the last 15 years. The updated search was carried out for articles published up to the end of November 2024.

    Search strategy

    The search strategy was designed with the support from a subject librarian.

    Various keywords and search terms and their combinations that define ‘primary healthcare’, ‘family doctor’, ‘digital tool’ and ‘triage’ were used. The search terms were combined using Boolean strategies of ‘AND’ and ‘OR’. The search was limited only to studies published in the last 20 years, and no other restriction criteria were applied. The search terms and strategy were amended as required when using different search databases. Medline (OvidSP), Embase (OvidSP), Cumulative Index to Nursing and Allied Health Literature, Web of Science and Scopus search databases were used in this review. Search terms for all five databases can be seen in online supplemental appendix 2. Reference hand searches were conducted for all included full texts. Additionally, citations in relevant systematic reviews were searched.

    Data management and screening

    Results from searches were imported and managed in Rayyan software. The articles were de-duplicated before study selection.

    To identify studies that potentially met the inclusion criteria, all titles and abstracts of the references were screened by AP, and a second screening was carried out by three reviewers (PP, DV and MC) independently. Consensus was reached by discussion or via input of a third reviewer where necessary. At the second stage, the full text of the articles was assessed by two reviewers (AP and PP) independently. Subsequently, the reviewers discussed to resolve any discrepancies. The final list of included studies was identified following the two stages of screening.

    The updated search results were screened by titles and abstracts by AP, and a second screening was carried out by two reviewers (HA and JP). Full-text screening was assessed by two reviewers (AP and JP). All discrepancies were discussed and resolved.

    The data from the included studies were extracted using a customised data extraction form that includes study details (author, country and setting where the study was carried out, design of study, date of publication), participant characteristics (type of healthcare staff using the tools), inclusion and exclusion criteria of the participants to the research, details of the intervention tools (tool name, country of use) and comparator if relevant, relevant outcomes and sources of funding. This was completed by AP, and these were checked for accuracy by PP.

    Backward and forward citation checking was also used.

    Risk of bias and quality assessment

    Quality assessment of studies that meet the inclusion criteria was carried out using the Mixed Methods Appraisal Tool V.2018.23 The tool was developed to suit the assessment of studies of different designs (qualitative, quantitative and mixed methods) for the quality assessment. The quality of the studies was used for contextualisation of the results only. Critical appraisal was performed by AP, and a proportion was checked for accuracy by PP, with any disagreements resolved via discussion.

    Strategy for data synthesis

    The results of data extraction and critical appraisal are summarised and analysed using narrative synthesis presented using tables and text. This approach was chosen because the included studies were heterogeneous in design, participant type (patients, primary healthcare staff, administrative staff) and outcomes.24

    Based on the data extracted, the included studies were categorised and analysed using the prespecified outcome groupings. Where studies included varying elements, only relevant findings and key themes were extracted and analysed in the review.

    Studies were categorised based on type (quantitative, qualitative and mixed methods) and outcome measures (patient outcomes, clinician outcomes and healthcare system outcomes). Findings were summarised using key themes identified across different studies.

    Results

    The initial search resulted in 10 145 records. After removing duplicates, there were 6825 records screened at the title and abstract stage, and 86 records were included for full-text screening, out of which 14 studies were included in the review. The updated search resulted in 3462 records. After removing duplicates, there were 2344 records screened at the title and abstract stage, and 23 records were included for full-text screening, out of which no studies were included in the review. See figure 1 for the PRISMA flowchart.

    Figure 1
    Figure 1

    PRISMA flow chart. Initial search for literature published between 1 January 2002 and 31 December 2022 and updated search for literature published up to 24 November 2024.*All 56 (first search) and seven (second search) excluded articles based on the intervention are articles that discuss interventions that did not meet the specific inclusion criteria, or met at least one of the exclusion criteria (not primary care tool, used to assess specific symptom (eg, dermatology), digital symptom checker, screening or providing likely diagnosis, do not inform triage by human clinician, provide access to direct consultation (without human triage) or were under development). PRISMA, Preferred Reporting Items for Systematic Reviews and Meta Analyses.

    The design of studies included qualitative (n=5),9 16 25–27 mixed methods (n=6)28–33 and quantitative (n=3).34–36 Qualitative designs included semistructured interviews, case studies and focus groups. Quantitative studies included analysis of survey responses and analysis of routinely collected data such as electronic healthcare records, log data from tool use, website analytics data and publicly available data.

    Studies were mainly conducted in the UK (n=9),16 26–30 32–34 with small numbers from Sweden (n=3)9 25 31 and Spain (n=2).35 36 Studies from the UK were mostly conducted in the southwest region of England (n=6)16 26–28 30 32 34 with one study in Scotland.29 One study was conducted in the UK Defence Primary Healthcare services.33

    Most studies were examining identifiable online triage platforms: eConsult (n=7),16 28–30 32–34 eConsulta (n=2),35 36 Flow (n=1)9 and Digital Primary Healthcare Service (n=1),31 while others involved ‘unnamed’ platforms (n=3).25–27 The summary of the included studies is in online supplemental appendix 3.

    No studies were included with clinical decision-making as the main outcome. Outcomes relating to clinical decision-making were within studies examining the wider use of online triage tools, and the outcomes relating to clinical decision-making are reported here. Outcomes examined in the studies were categorised into three groups: patient clinical outcomes (n=9),9 16 26 28–30 32–34 primary care practitioner experience outcomes (n=11)9 16 25–33 and healthcare system outcomes (n=14).9 16 25–36 The outcomes and key themes are presented in table 1.

    View this table:
    Table 1

    Findings identified in the included studies

    Nine out of the 14 included studies were checked for quality score accuracy by PP. There were disagreements on seven of the quality scores. Disagreements were resolved via discussion. Following the assessment of study quality, most studies were given a high (n=5)9 16 25 26 34 or medium quality rating (n=7),28 30–33 35 36 while two studies were rated as low quality.27 29 Higher quality studies were mostly qualitative while mixed methods and quantitative studies were more variable. Reasons for lower quality included lack of information on qualitative and quantitative components, integration process and result interpretation in mixed methods studies and lack of information on confounding factors in quantitative studies. One qualitative study was downgraded from medium to low quality as there was not enough information on recruitment strategy and data collection. The results of the quality assessment are included in online supplemental appendix 4.

    Findings relating to patient clinical outcomes

    Nine studies discussed patient clinical outcomes relating to clinical decision-making. The findings relating to patient clinical outcomes were as follows: when responding to an online request from a patient, clinicians are comfortable to adjust prescription appropriately for an ongoing issue, for example, in response to change in blood pressure (n=5),16 28 29 33 34 faster clinical decision time in response to patient concern (n=4),9 28 33 34 delay in urgent referrals (eg, because patient downplaying their symptoms to avoid calling emergency services) (n=3)28 29 32 and decisions to give patients unnecessary access to urgent primary care appointments were influenced by patients using the systems (n=4).9 29 32 33

    Findings of primary care practitioner experience outcomes

    11 studies discussed the experiences of primary care practitioners and their confidence in decision-making when using online triage tools. Five negative impacts on clinical decision-making were identified in the studies: challenged clinical decision-making without in-person appointment cues (eg, inability to probe the patient for further information or reading body language) (n=3),9 16 33 difficulties in identifying patient key concerns due to one of two reasons: incomplete information given by the patient completing the form or the clinician finding it difficult to identify. patient expectations (n=5),9 16 25 30 31 hesitation in prescribing specific drugs such as antibiotics and addictive drugs (n=2),31 33 limitations due to reliance on textual information provided by the patients (n=5)16 26 28 31 32 and lack of quality of information or mismatched free text information with the disease form chosen by the patient, leading clinicians to hesitate to make any clinical decision without calling the patient or arranging an appointment (n=6).16 25–27 30 31 Two positive impacts were identified in the studies: patients are able to send photos for skin problems, enabling clinicians to make decisions faster (n=4),9 25 31 33 and clinicians are more confident in managing patient requests (eg, as they have time to read and plan appropriate action) (n=3).9 16 28 29

    Findings of healthcare system outcomes

    The findings of healthcare system outcomes in relation to their impact on clinical decision-making were divided into two groups: workload (n=13)9 16 25–31 33–36 and frequency of primary care appointments (n=10).9 16 25 26 28 29 31–34

    Studies suggested that clinical decision-making is affected by clinicians’ workload, as freeing up time may allow more focused clinical decisions on complex cases or cases with high priority. From this perspective, there are benefits from using the online triage tools on workload: reduction in face-to-face and telephone appointments, particularly in interactions with limited clinical value such as management of test results, repeat prescriptions and sick notes (n=3),16 35 36 the forms replace short telephone appointments such as prescription review (n=3),9 29 33 reduction of administrative burden from the clinicians as some of the clinical decisions are instructed to non-clinical administrators to complete the consultation and communicated with the patient (n=9)9 16 26–31 34 and empowering the patients to take a more active role, leading to reduce some of the workload from the clinicians (eg, patients are able to research their concerns and submit articulate complaints, freeing up time for the clinicians to focus on making clinical decisions. Also, some patients, such as patients with mental health issues, are able to spend time phrasing their complaints in a private setting (n=6).9 25–28 33

    There were negative impacts of triage tool use on clinicians’ workload, putting pressure on clinicians and reducing the time spent on making clinical decisions. The platforms added extra stages in the workflow (reading the form, follow-up (telephone/in person) and then transferring information from the form to patients’ records) (n=8).9 16 26–28 30 33 34 Moreover, patients raising the same issue concurrently using the tools and other routes are leading to duplicate work where different general practice clinicians are dealing with the same issue. This led to multiple clinical decisions communicated to the patient at once or, in some cases, leaving patient concerns unattended where clinicians perceived that a clinical decision had been taken via another route (n=3).9 25 30 Also, the triage tools incorrectly highlighted cases as urgent, leading to escalated clinical decision for a minor issue (n=1).26

    There are additional benefits relating to the frequency of primary care appointments. The tools provided continuity of access for patients with long-term conditions and frequent attenders (potentially freeing up appointments), improving related clinical decisions (n=2).25 33 However, the tools are perceived by clinicians as an additional point of access, as the type and nature of concerns raised using the online tools differ from traditional appointments, such as patients who are familiar with their condition or need straightforward advice from a doctor tend to contact their general practice using the online tools. Additionally, patients who might be embarrassed or anxious to discuss their health issues in person were able to articulate their concerns using the online tools. Therefore, clinicians expect to make clinical decisions differently from what they are used to in traditional appointments (n=3).16 28 33

    Discussion

    This systematic review evaluated the evidence on how patient-facing online triage tools in primary care affect clinical decision-making. 14 studies were included covering a range of different designs, populations and tools.

    Summary of the main findings

    The review examined patient clinical outcomes, primary care practitioners’ experience and healthcare system outcomes in relation to clinical decision-making.

    For instance, clinicians reduced the time to make clinical decisions in response to patient concerns, such as issuing repeat prescriptions remotely. On the other hand, some clinicians were concerned with patients learning to game the system to access urgent appointments or to avoid being forwarded to emergency services.

    Moreover, primary care clinicians felt confident in managing patients’ requests as they spent more time planning their decision and would have access to additional information such as patient records or a photo attached to the form.

    Additionally, clinicians’ time is freed up from face-to-face and telephone appointments with limited clinical value. However, online triage is perceived as an additional step in the workflow of clinical decision-making, as a large number of online triages lead to telephone or face-to-face consultation.

    Comparison with the previous literature

    This systematic review found that clinical decision-making is challenging without in-person cues, as it was limited to information provided by the patients in the online form. Similarly, previous literature on using online consultation tools in UK primary care has shown that missing face-to-face interaction meant that clinicians did not find remote consultations as adequate replacement to in-person consultations.12 21 37 In this systematic review, it was found that clinicians have reported difficulty in identifying patient concerns and expectations due to incomplete information shared on the online form. This agrees with previous studies on using online consultation tools in UK primary care that showed clinicians reporting concerns on the increased risk of missing information.11 12 21 37 While the findings in this systematic review were similar, it should be noted that the online consultation tools examined in previous studies had different functionalities compared with those discussed in this review.

    Although the previous systematic review by Darley et al20 examined the broader context of using the online consultation tools, it found limited evidence on how they influence clinical decision-making. Darley et al’s review highlighted an evidence gap as the studies reviewed did not provide information on the process of clinical decision-making. There were some concerns that were discussed in Darley et al’s review, in particular, in relation to the loss of information compared with in-person or telephone consultations, which may lead to misdiagnosis. Additional concerns were related to the ability of the tools to identify when a patient is not suitable for an online query.

    Furthermore, this review found that clinicians benefited from the additional details sent by the patient using the online form, for example, attaching photos to the form, and there was increased confidence in managing requests as the clinician has time to read and plan appropriate action. On the contrary, Mold et al’s systematic review21 found limited evidence that clinicians with access to patient notes together with the information shared by the patient using online means such as secure email and messaging or video links improved the potential for shared decision-making, where the patient can influence clinical decisions.

    Strengths and limitations

    This is the first systematic review to analyse outcomes relating to clinical decision-making from triaging patients who completed an online form to contact their primary care service. A strength of this review is that it evaluates outcomes of tool usage across different countries. However, it was limited to studies published in English, and this may have led to missing some evidence from countries with universal healthcare coverage that might have been early adopters of the primary care patient online triage (such as the Netherlands and the Scandinavian countries). As the review only included published studies, there is a risk that relevant studies may have been excluded where these are published in the grey literature.

    The search strategy for this systematic review was carefully planned with input from subject matter experts and an expert librarian. This included carefully planning the search terms, synonyms, related concepts and Medical Subject Headings (MeSH) terms based on the research question. Moreover, the search was conducted on five databases.

    The application of patient-facing online triage is very recent, leading to a relatively small number of included studies, and this was particularly driven by the focus on the clinical decision-making, which has not been sufficiently studied until now. Some of the included studies are pilot and feasibility studies (eg, one study was based on a master’s dissertation with a very small sample size27 in which the piloted sites ceased offering the tools to their patients at the end of the study). The recent adoption of the tools might indicate that the benefits are not observed long enough to evidence their sustainability, and the implementation was not sufficiently long to identify solutions to some of the issues resulting from using the tools.

    The majority of the included studies were evaluating one tool (eConsult) and based in the UK, which limits generalisability to primary care in other healthcare systems. Additionally, a few studies included tools without clear description of their functionality (such as 36 and 35), limiting the comparability of evaluated outcomes.

    Implications (for research and practice)

    Patient-facing online triage tools are becoming more widely used, and policymakers are creating the conditions to make their use the standard for patients contacting their general practice.38 This review has identified impacts on clinicians’ experience when using outputs from patient-facing online triage tools. Therefore, the findings from this review can be used to increase awareness of clinicians on how their decision-making processes are likely to differ. Additionally, developers can use the findings to improve the usability of the tools’ outputs.

    None of the identified studies were directly about the clinical decision-making process but rather examined outcomes related to the clinical decision-making process. Therefore, gaps in the literature have been identified, particularly a need to assess the impact of using patient-facing online triage tools in general practice on the process of clinical decision-making and qualitative analysis of clinicians’ experiences in making clinical decisions when using outputs from the tools. Additionally, there should be an analysis of large patient datasets (particularly linking outputs from the tools and subsequent referral to other health and care services) to provide data-driven evidence.

    This review highlights that associations between patient characteristics (such as patients with mental health issues or patients able to do their own research before completing a form) and the experience of clinicians and the quality of clinical decision-making have not been explored in depth. As the tools continue to be used by patients and clinicians, data will be available to examine the quality of clinical decisions made using outputs from the tools with different medical conditions (eg, cancer and mental health). Additionally, with more detailed data, tool developers would be able to identify recurring clinical decisions such as responding to routine enquiries, test results and repeat prescriptions. Therefore, developers might be able to integrate the straightforward elements of the clinical decisions into the tool reports to the clinician.

    None of the included studies explored the interoperability of the tools with general practice electronic health records. It is expected that some elements of interoperability will enable tools to report a more comprehensive summary of the patient concerns and any relevant patient history from their records. Research will help developers to identify areas from patient history that are needed for clinical decision-making in relation to different patient concerns.

    Lastly, this review included a few studies integrating findings from routine data with qualitative research. Mixed methods research will help to better understand clinician experiences and support any qualitative findings with evidence from data.

    Data availability statement

    All data relevant to the study are included in the article or uploaded as supplementary information.

    Data availability statement

    Relevant data are included in online supplemental tables.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    Acknowledgments

    The authors would like to thank Samantha Johnson (Academic Support Librarian, University of Warwick) for support with developing the search strategy. The authors would like to thank Prashanth Pandiyan, Daniela Valdes and Mike Cooke for their help in conducting screening. Additionally, the authors would like to thank Prashanth Pandiyan for his contribution in data extraction and quality assessment.

    References

    1. ↵
      Advice on How Tp Establish a Remote “Total Triage” Model in General Practice Using Online Consultations (Version 3). NHS England, 2020.
    2. ↵
      1. Campbell JL,
      2. Fletcher E,
      3. Britten N, et al
      . The clinical effectiveness and cost-effectiveness of telephone triage for managing same-day consultation requests in general practice: a cluster randomised controlled trial comparing general practitioner-led and nurse-led management systems with usual care (the ESTEEM trial). Health Technol Assess 2015;19:1–212. doi:10.3310/hta19130
      OpenUrlCrossRefPubMed
    3. ↵
      1. Campbell JL,
      2. Fletcher E,
      3. Britten N, et al
      . Telephone triage for management of same-day consultation requests in general practice (the ESTEEM trial): a cluster-randomised controlled trial and cost-consequence analysis. The Lancet 2014;384:1859–68. doi:10.1016/S0140-6736(14)61058-8
      OpenUrl
    4. ↵
      1. Newbould J,
      2. Abel G,
      3. Ball S, et al
      . Evaluation of telephone first approach to demand management in English general practice: observational study. BMJ 2017;358:j4197. doi:10.1136/bmj.j4197
    5. ↵
      1. Hobbs FDR,
      2. Bankhead C,
      3. Mukhtar T, et al
      . Clinical workload in UK primary care: a retrospective analysis of 100 million consultations in England, 2007–14. The Lancet 2016;387:2323–30. doi:10.1016/S0140-6736(16)00620-6
      OpenUrlCrossRef
    6. ↵
      The NHS Long Term Place. NHS England: London, 2019.
    7. ↵
      Fit for the Future: A Vision for General Practice. London: Royal College of General Practice, 2019.
    8. ↵
      1. Gerada C, et al
      . The 2022 GP: A Vision for General Practice in the Future NHS. London: Royal College of General Practitioners, 2013.
    9. ↵
      1. Eldh AC,
      2. Sverker A,
      3. Bendtsen P, et al
      . Health Care Professionals’ Experience of a Digital Tool for Patient Exchange, Anamnesis, and Triage in Primary Care: Qualitative Study. JMIR Hum Factors 2020;7:e21698. doi:10.2196/21698
    10. ↵
      1. Smits M,
      2. Hanssen S,
      3. Huibers L, et al
      . Telephone triage in general practices: A written case scenario study in the Netherlands. Scand J Prim Health Care 2016;34:28–36. doi:10.3109/02813432.2016.1144431
      OpenUrlCrossRefPubMed
    11. ↵
      1. Brant H,
      2. Atherton H,
      3. Ziebland S, et al
      . Using alternatives to face-to-face consultations: a survey of prevalence and attitudes in general practice. Br J Gen Pract 2016;66:e460–6. doi:10.3399/bjgp16X685597
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Salisbury C,
      2. Murphy M,
      3. Duncan P
      . The Impact of Digital-First Consultations on Workload in General Practice: Modeling Study. J Med Internet Res 2020;22:e18203. doi:10.2196/18203
    13. ↵
      1. Rodrigues D,
      2. Kreif N,
      3. Saravanakumar K, et al
      . Formalising triage in general practice towards a more equitable, safe, and efficient allocation of resources. BMJ 2022;377:e070757. doi:10.1136/bmj-2022-070757
    14. ↵
      1. Bate L,
      2. Hutchinson A,
      3. Underhill J, et al
      . How clinical decisions are made. Brit J Clinical Pharma 2012;74:614–20. doi:10.1111/j.1365-2125.2012.04366.x
      OpenUrl
    15. ↵
      1. Brady M,
      2. Northstone K
      . Remote clinical decision-making: a clinician’s definition. Emerg Nurse 2017;25:24–8. doi:10.7748/en.2017.e1661
      OpenUrl
    16. ↵
      1. Banks J,
      2. Farr M,
      3. Salisbury C, et al
      . Use of an electronic consultation system in primary care: a qualitative interview study. Br J Gen Pract 2018;68:e1–8. doi:10.3399/bjgp17X693509
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. North F,
      2. Richards DD,
      3. Bremseth KA, et al
      . Clinical decision support improves quality of telephone triage documentation--an analysis of triage documentation before and after computerized clinical decision support. BMC Med Inform Decis Mak 2014;14:1–10:20. doi:10.1186/1472-6947-14-20
      OpenUrlCrossRefPubMed
    18. ↵
      1. Jones D,
      2. Dunn L,
      3. Watt I, et al
      . Safety netting for primary care: evidence from a literature review. Br J Gen Pract 2019;69:e70–9. doi:10.3399/bjgp18X700193
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Edwards PJ,
      2. Ridd MJ,
      3. Sanderson E, et al
      . Safety netting in routine primary care consultations: an observational study using video-recorded UK consultations. Br J Gen Pract 2019;69:e878–86. doi:10.3399/bjgp19X706601
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Darley S,
      2. Coulson T,
      3. Peek N, et al
      . Understanding how the design and implementation of online consultations influence primary care outcomes: systematic review of evidence with recommendations for designers, providers, and researchers. Primary Care Research [Preprint] 2022. doi:10.1101/2022.02.21.22271185
    21. ↵
      1. Mold F,
      2. Hendy J,
      3. Lai Y-L, et al
      . Electronic Consultation in Primary Care Between Providers and Patients: Systematic Review. JMIR Med Inform 2019;7:e13042. doi:10.2196/13042
    22. ↵
      1. Moher D,
      2. Shamseer L,
      3. Clarke M, et al
      . Preferred Reporting Items for Systematic Review and Meta-Analysis. Syst Rev 2015;1–9. doi:10.1186/2046-4053-4-1
    23. ↵
      1. Hong QN,
      2. Fàbregues S,
      3. Bartlett G, et al
      . The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. EFI 2018;34:285–91. doi:10.3233/EFI-180221
      OpenUrlCrossRef
    24. ↵
      1. Popay J,
      2. Roberts H,
      3. Sowden A, et al
      . Guidance on the conduct of narrative synthesis in systematic reviews. A product from the ESRC methods programme Version. 2006;1:b92. doi:10.13140/2.1.1018.4643
    25. ↵
      1. Eriksson P,
      2. Hammar T,
      3. Lagrosen S, et al
      . Digital consultation in primary healthcare: the effects on access, efficiency and patient safety based on provider experience; a qualitative study. Scand J Prim Health Care 2022;40:498–506. doi:10.1080/02813432.2022.2159200
      OpenUrl
    26. ↵
      1. Turner A,
      2. Morris R,
      3. Rakhra D, et al
      . Unintended consequences of online consultations: a qualitative study in UK primary care. Br J Gen Pract 2022;72:e128–37. doi:10.3399/BJGP.2021.0426
      OpenUrlAbstract/FREE Full Text
    27. ↵
      1. Casey M,
      2. Shaw S,
      3. Swinglehurst D
      . Experiences with online consultation systems in primary care: case study of one early adopter site. Br J Gen Pract 2017;67:e736–43. doi:10.3399/bjgp17X693137
      OpenUrlAbstract/FREE Full Text
    28. ↵
      1. Carter M,
      2. Fletcher E,
      3. Sansom A, et al
      . Feasibility, acceptability and effectiveness of an online alternative to face-to-face consultation in general practice: a mixed-methods study of webGP in six Devon practices. BMJ Open 2018;8:e018688. doi:10.1136/bmjopen-2017-018688
    29. ↵
      1. Cowie J,
      2. Calveley E,
      3. Bowers G, et al
      . Evaluation of a Digital Consultation and Self-Care Advice Tool in Primary Care: A Multi-Methods Study. Int J Environ Res Public Health 2018;15:896. doi:10.3390/ijerph15050896
    30. ↵
      1. Farr M,
      2. Banks J,
      3. Edwards HB, et al
      . Implementing online consultations in primary care: a mixed-method evaluation extending normalisation process theory through service co-production. BMJ Open 2018;8:e019966. doi:10.1136/bmjopen-2017-019966
    31. ↵
      1. Johansson A,
      2. Larsson M,
      3. Ivarsson B
      . General Practitioners’ Experiences of Digital Written Patient Dialogues: A Pilot Study Using a Mixed Method. J Prim Care Community Health 2020;11:2150132720909656. doi:10.1177/2150132720909656
      OpenUrl
    32. ↵
      1. Jones RB,
      2. Tredinnick-Rowe J,
      3. Baines R, et al
      . Use and usability of GP online services: a mixed-methods sequential study, before and during the COVID-19 pandemic, based on qualitative interviews, analysis of routine eConsult usage and feedback data, and assessment of GP websites in Devon and Cornwall, England. BMJ Open 2022;12:e058247. doi:10.1136/bmjopen-2021-058247
    33. ↵
      1. Willman AS
      . Evaluation of eConsult use by Defence Primary Healthcare primary care clinicians using a mixed-method approach. BMJ Mil Health 2023;169:e39–43. doi:10.1136/bmjmilitary-2020-001660
      OpenUrlAbstract/FREE Full Text
    34. ↵
      1. Edwards HB,
      2. Marques E,
      3. Hollingworth W, et al
      . Use of a primary care online consultation system, by whom, when and why: evaluation of a pilot observational study in 36 general practices in South West England. BMJ Open 2017;7:e016901. doi:10.1136/bmjopen-2017-016901
    35. ↵
      1. López Seguí F,
      2. Walsh S,
      3. Solans O, et al
      . Teleconsultation Between Patients and Health Care Professionals in the Catalan Primary Care Service: Message Annotation Analysis in a Retrospective Cross-Sectional Study. J Med Internet Res 2020;22:e19149. doi:10.2196/19149
    36. ↵
      1. López Seguí F,
      2. Vidal-Alaball J,
      3. Sagarra Castro M, et al
      . General Practitioners’ Perceptions of Whether Teleconsultations Reduce the Number of Face-to-face Visits in the Catalan Public Primary Care System: Retrospective Cross-Sectional Study. J Med Internet Res 2020;22:e14478. doi:10.2196/14478
    37. ↵
      1. Murphy M,
      2. Scott LJ,
      3. Salisbury C, et al
      . Implementation of remote consulting in UK primary care following the COVID-19 pandemic: a mixed-methods longitudinal study. Br J Gen Pract 2021;71:e166–77. doi:10.3399/BJGP.2020.0948
      OpenUrlAbstract/FREE Full Text
    38. ↵
      1. England N
      . Online consultations. NHS App Guidance for GP practices, 2023. Available: https://digital.nhs.uk/services/nhs-app/nhs-app-guidance-for-gp-practices/guidance-on-nhs-app-features/online-consultations [Accessed 29 Feb 2024].

    Footnotes

    • X @h_atherton

    • Contributors AP developed the review protocol, with the support of HA, Jeremy Dale, GAA and Benedict Hayhoe. AP conducted searches, screening, data extraction and quality assessment. AP conducted the narrative synthesis with support from HA. JP and GAA reviewed and revised manuscript and approved the final version. AP is the guarantor for this systematic review.

    • Funding This systematic review is part of a PhD that is funded through University of Warwick, Warwick Medical School in collaboration with an industrial partner: eConsult (https://eConsult.net/). HA, GAA and JP are supervisors of AP’s PhD study but are not in receipt of any funding for this and are not based at the same institution.

    • Competing interests The lead author, AP, receives a PhD studentship via a Warwick Industrial Fellowship, in conjunction with eConsult Ltd. eConsult funds 50% of the studentship. They are not involved in the design or conduct of the research (beyond specifying the broad research area), and analysis is conducted independently of eConsult. All other authors have no completing interest to declare.

    • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.