Diagnosing malignant melanoma in ambulatory care: a systematic review of clinical prediction rules
==================================================================================================

* Emma Harrington
* Barbara Clyne
* Nieneke Wesseling
* Harkiran Sandhu
* Laura Armstrong
* Holly Bennett
* Tom Fahey

## Abstract

**Objectives** Malignant melanoma has high morbidity and mortality rates. Early diagnosis improves prognosis. Clinical prediction rules (CPRs) can be used to stratify patients with symptoms of suspected malignant melanoma to improve early diagnosis. We conducted a systematic review of CPRs for melanoma diagnosis in ambulatory care.

**Design** Systematic review.

**Data sources** A comprehensive search of PubMed, EMBASE, PROSPERO, CINAHL, the Cochrane Library and SCOPUS was conducted in May 2015, using combinations of keywords and medical subject headings (MeSH) terms.

**Study selection and data extraction** Studies deriving and validating, validating or assessing the impact of a CPR for predicting melanoma diagnosis in ambulatory care were included. Data extraction and methodological quality assessment were guided by the CHARMS checklist.

**Results** From 16 334 studies reviewed, 51 were included, validating the performance of 24 unique CPRs. Three impact analysis studies were identified. Five studies were set in primary care. The most commonly evaluated CPRs were the ABCD, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter (ABCD) dermoscopy rule (at a cut-point of >4.75; 8 studies; pooled sensitivity 0.85, 95% CI 0.73 to 0.93, specificity 0.72, 95% CI 0.65 to 0.78) and the 7-point dermoscopy checklist (at a cut-point of ≥1 recommending ruling in melanoma; 11 studies; pooled sensitivity 0.77, 95% CI 0.61 to 0.88, specificity 0.80, 95% CI 0.59 to 0.92). The methodological quality of studies varied.

**Conclusions** At their recommended cut-points, the ABCD dermoscopy rule is more useful for ruling out melanoma than the 7-point dermoscopy checklist. A focus on impact analysis will help translate melanoma risk prediction rules into useful tools for clinical practice.

*   PRIMARY CARE
*   Melanoma
*   Clinical prediction rules
*   Systematic review

### Strengths and limitations of this study

*   The main strengths of this review are the use of broad inclusion criteria, the systematic search of multiple databases not limited by language, use of the CHARMS checklist to assess methodological quality, pooling data from a broad range of studies to enhance generalisability and the use of a broad definition of primary care to account for the variation in primary care services and access internationally. Quality assessment criteria were used to assess risk of bias and the majority of studies were at low risk in relation to the randomisation procedure and monitoring of loss to follow-up.

*   A large proportion of studies did not provide sufficient information and data to perform stratified meta-analysis according to different levels of risk.

*   Current research shows that dermoscopic clinical prediction rules (CPRs) may be a useful tool for primary care physicians prioritising appropriate referrals for higher risk patients and adopting a watchful waiting strategy in lower risk patients but future impact analysis research is necessary to establish their impact on patient outcomes.

## Introduction

The incidence of malignant melanoma in most developed countries has been steadily rising (faster than other cancer types) in recent decades.1 ,2 Increases in the age-standardised incidence of at least 4–6% per annum have been reported internationally in many fair-skinned populations including Australia, the USA and most of Europe.3–5 Simultaneously, there has been a significant rise in overall 5-year survival in melanoma patients, largely attributable to earlier detection and diagnosis of thinner tumours.6 While the majority of patients may survive melanoma, the disease has a significant impact on patient quality of life7 and healthcare expenditure, with the average annual total treatment costs for melanoma in the USA increasing to US$3.3 billion in 2011.8 Melanoma is potentially preventable since a significant risk factor, exposure to ultraviolet (UV) radiation, is modifiable.9 However, other risk factors (eg, number naevi, eye and hair colour, freckles, familial history and genetic predisposition) also play an important role in the risk of developing melanoma.10 ,11

Early detection followed by curative surgery greatly improves melanoma prognosis. However, early detection may be affected by the challenging natures of differential diagnosis of pigmented lesions. Particularly in primary care where the evaluation of suspected skin lesions is imposing an increasing burden due to rising incidences of skin cancer.12 It has been suggested that primary care practitioners' skills of diagnosing skin lesions could be improved.13 A number of clinical prediction pules (CPRs) and computer-assisted diagnostic tools have been developed to assist in distinguishing malignant melanoma from benign pigmented skin lesions. The UK National Institute for Clinical Excellence (NICE) guidelines advise against routine use of computer-assisted diagnostic tools in the initial evaluation of a pigmented skin lesion (NICE guidelines) and promote use of the weighted 7-point checklist in primary care to guide referral (NG12). When used by dermatologists for the diagnosis of melanoma, certain CPRs have demonstrated high sensitivity and specificity.6 Although each CPR has its own unique elements, there is significant overlap in terms of their content (see online supplementary appendix 1), and while their use is promoted, it is unclear which rules are most suitable for use in primary care.

### supplementary appendices

[[bmjopen-2016-014096supp_appendices.pdf]](pending:yes)

CPRs may be for use in clinical (ie, naked eye) examination, or in conjunction with dermoscopy. Dermoscopy, dermatoscopy or epiluminescent microscopy refers to the examination of pigmented skin lesions using surface microscopy.14 ,15 The use of dermoscopy, primarily by dermatologists, has been found to increase diagnostic accuracy compared with naked-eye inspection, as it allows the visualisation of features that are not visible to the naked eye.14–16 However, the effectiveness of dermoscopy depends on clinical experience and training. Dermatologists with formal training in dermoscopy have higher melanoma detection rates compared with untrained dermatologists and primary care physicians.16–18

As primary care or ambulatory care physicians are frequently and increasingly confronted with the care of skin lesions suspected of malignancy,12 it is essential to identify tools to aid primary care practitioners to differentiate patients with clinically significant lesions, requiring referral, from those who can be treated and monitored in primary care. The aim of this study was to perform a systematic review of CPRs for the diagnosis of malignant melanoma, to evaluate their diagnostic accuracy in primary care and specialist outpatient settings, among patients with a pigmented skin lesion. Secondary aims were to review studies that have examined the implementation of CPRs in clinical practice through impact analysis studies.

## Methods

The protocol for this systematic review was published on PROSPERO (CRD42015020898) and was conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.19

### Search strategy and data sources

A systematic literature search was conducted (May 2015) including the following databases: PubMed, EMBASE, PROSPERO, CINAHL, the Cochrane Library and SCOPUS, using combinations of the following keywords and MeSH terms: melanoma/diagnosis, melanoma, prediction, score, model, decision, sensitivity, specificity, validate, derived. Hand searches of references of retrieved full-text articles and key author searches supplemented the search. No date or language limits were imposed.

### Study selection

All articles were initially screened for inclusion according to title and abstract by two reviewers (NW, EH). Full-text articles of studies considered eligible for inclusion were independently read by both reviewers, with any disagreements resolved by a third reviewer (BC).

#### Validation studies

Validation studies were eligible for inclusion if they met the following criteria; 

1.  *Population:* Adults (age ≥18 years) with a pigmented skin lesion in ambulatory care settings in general practice/family medicine, dermatology, plastic surgery and other relevant specialties.

2.  *Risk:* Derivation and/or validation of a CPR for melanoma diagnosis to aid decision making about referral or investigation of a pigmented skin lesion. CPRs were defined as ‘a clinical tool that quantifies the individual contributions that various components of the history, physical examination and investigations make toward the diagnosis, prognosis or likely response to treatment in a patient’.

3.  *Comparison:* Usual clinical judgement for decision making about referral or investigation OR another CPR for melanoma diagnosis.

4.  *Primary outcome:* Performance of a CPR for predicting diagnosis of malignant melanoma (in terms of sensitivity, specificity, negative predictive values and positive predictive values).

Observational study designs (eg, cohort, cross-sectional, case–control) were included. Studies were excluded where they had undergone derivation only, reported individual predictors only, or used computer-assisted diagnostic tools, following the NICE guideline recommendation against the routine use of computer-assisted diagnostic tools.20

#### Impact analysis

The following study designs were included for impact analysis: (cluster) randomised controlled trials (RCTs), controlled before–after studies or interrupted time series studies. We excluded uncontrolled study designs. We included studies where a melanoma CPR was used to predict melanoma compared with usual care in the clinical setting. The outcomes of interest included physician behaviour, process of care, patient outcomes and/or cost-effectiveness. A requirement for inclusion was that the CPR comprised the entire intervention. Studies where the CPR was implemented as part of a broader guideline, protocol or decision aid were excluded. Studies that used a CPR to determine eligibility for trial inclusion but were not part of the intervention were also excluded.

### Data extraction

Data were extracted by four reviewers (LA, HB, HS, EH) using a data form based on the CHARMS checklist.21 Data extracted included study design and setting, patient demographics and inclusion criteria, CPR name, CPR type (clinical or dermoscopic), predictive accuracy of the CPR (sensitivity/specificity) and for impact analysis, the impact on the primary outcome.

### Critical appraisal of studies

Two reviewers (EH, NW) critically appraised included studies using the CHARMS checklist, developed to provide guidance on data extraction and critical appraisal of prediction modelling studies.21 The checklist contains 11 domains of critical appraisal. The methodological quality of each study was independently evaluated by two reviewers and by a third reviewer if consensus was not reached. The methodological quality of each impact analysis study was also independently assessed, using an appropriate quality assessment checklist. RCTs were assessed using the Cochrane risk of bias tool and controlled before–after studies were evaluated using Cochrane criteria for these study designs.22

### Statistical analysis

Statistical analysis was conducted using Stata V.12 (StataCorp, College Station, Texas, USA), in particular the metandi and midas commands. For each CPR, a standard cut-point was identified (table 1). From each included study we extracted (where available) the numbers of true positives, false positives, true negatives, false negatives, sensitivity and specificity and their corresponding 95% CIs. Where sensitivity/specificity for more than one observer was reported, the mean value was included in the analysis. Studies were grouped for analysis by CPR type (ie, clinical or dermoscopic). Summary estimates of sensitivity and specificity and their corresponding 95% CIs were calculated using the bivariate random effects model (midas). The bivariate model has the benefits of being easily interpretable, is technically straightforward to undertake and takes into account the sample and heterogeneity beyond chance between studies.23

View this table:
[Table 1](http://bmjopen.bmj.com/content/7/3/e014096/T1)

Table 1 
CPRs identified for inclusion with cut-points for identification of melanoma

Individual and summary estimates of sensitivity and specificity were plotted on a hierarchical summary receiver operating characteristic (HSROC) graph. This approach incorporates sensitivity and specificity, while taking into account the correlation between the two.24 Sensitivity (true positive) was graphed on the y-axis and 1-specificity (false negative) on the x-axis. The 95% confidence region and the 95% prediction region were also plotted around the pooled estimates in order to depict the precision with which the pooled estimates were determined (confidence ellipse around the mean value) and to illustrate the amount of between-study variation (prediction ellipse).

## Results

### Study selection

The search strategy yielded a total of 25 816 articles. Of these 9481 were duplicates and 16 166 were deemed irrelevant based on title/abstract. The remaining 171 were reviewed in full with 51 meeting the inclusion criteria (see online supplementary appendix 2). From these, 24 unique melanoma CPRs were identified (table 1). Twelve papers reported derivation and validation studies, 36 were validation studies only and three were impact analyses.

### Summary of studies

Table 2 summarises the characteristics of the included studies. The majority (11, 22%) were conducted in Italy14 ,15 ,25–34 and ranged from an analysis of 40 lesions to 1580 lesions. From 13 studies providing information, mean age of included patients ranged from 36.7 to 53.25 ,28 ,31 ,35–44 From the 14 studies that reported gender, the proportion of males ranged from 22% to 60%.25 ,31 ,33 ,35–45 In total, 31 of the 50 studies were published in/or after 2000.14 ,25 ,28 ,29 ,31–37 ,42–44 ,46–62 Five studies were set in primary care,36 ,44 ,49 ,62 ,63 with the remainder undertaken in specialist outpatient settings.

View this table:
[Table 2](http://bmjopen.bmj.com/content/7/3/e014096/T2)

Table 2 
Characteristics of validation and impact analysis studies included

### Summary of CPRs identified

Of the 24 rules identified, four were clinical (ie, naked eye), 17 were dermoscopic and the remaining three used novel diagnostic technologies. The most commonly applied clinical CPR was the ABCDE rule (five studies),6 ,15 ,28 ,64 ,65 while for dermoscopy the most common were the Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter (ABCD) rule of dermoscopy (23 studies)14 ,25 ,26 ,29 ,31 ,32 ,39 ,42 ,43 ,47–49 ,52 ,53 ,57 ,65–70 and the 7-point checklist for dermoscopy (17 studies).14 ,25 ,26 ,29 ,35 ,37 ,42 ,43 ,46–50 ,52 ,56 ,57 ,59

Each of the elements included in the 24 rules identified are presented in table 3. All four clinical rules included the elements of diameter and colour variegation (table 3 and see online supplementary appendix 1). The most frequently included elements in the 17 dermoscopic rules were multiple colours (13 rules), asymmetry (12 rules) and streaks (10 rules) (table 3 and see online supplementary appendix 1).

View this table:
[Table 3](http://bmjopen.bmj.com/content/7/3/e014096/T3)

Table 3 
Comparison of elements in clinical prediction rules for malignant melanoma

### Methodological quality of validation studies

Based on the CHARMS checklist, the quality of included studies varied.21 All studies had weaknesses in study design and quality assessment was often hindered by poor reporting of methods. The studies had reasonable sample sizes and all provided adequate definitions of the outcome of interest. However, a number of important weaknesses were identified. None of the studies reported on missing data and key performance measures of model performance (eg, calibration) were often missing. Derivation studies typically reported information on model development, in terms of selection of candidate predictors, selection of predictors during modelling, and model evaluation. However, often the methods applied introduced a strong risk of bias, for example, a number of studies described splitting the original sample into a development and validation sample which is considered statistically inefficient and results in overfitting of the model.21 Full results of the quality assessment are shown in online supplementary appendix 3.

### Predictive accuracy of melanoma CPRs

The results for the most commonly applied CPRs, the ABCD rule and the 7-point checklist are presented here. The sensitivity and specificity of all rules identified (including the ABCDE clinical rule, the seven features for melanoma rule and Menzies dermoscopy for melanoma rule) are summarised in table 4.

View this table:
[Table 4](http://bmjopen.bmj.com/content/7/3/e014096/T4)

Table 4 
Sensitivity and specificity of all clinical and dermoscopy CPRs

### Clinical (naked eye) CPRs for melanoma diagnosis

Four studies validating the ABCDE clinical rule6 ,15 ,28 ,64 and one validating the ABCD clinical rule65 were included. There was insufficient data to conduct any meta-analysis. Rao *et al* reported a sensitivity of 0.84 and specificity of 0.78, for an unspecified cut-point.65

Six studies validating the original and revised 7-point checklist were included. There was insufficient data to conduct a meta-analysis. Of the four studies validating the original 7-point checklist (cut-point ≥3), three reported sensitivity (range 0.44–0.86, mean 0.70) and specificity (range 0.62–0.94, mean 0.74).40 ,41 ,44 Only one of the four studies validating the revised 7-point checklist (cut-point ≥1) reported sensitivity (0.92) and specificity (0.33) (table 4).44

### Dermoscopic CPRs for melanoma diagnosis

#### ABCD rule of dermoscopy

The ABCD rule of dermoscopy (also described as the ABCD rule of Stolz), was validated in 23 studies, 15 of which applied a cut-point of >4.75 (indicating a suspicious lesion) and six studies a cut-point of 5.45 (highly suggestive for melanoma). At a cut-point of >4.75, eight studies provided sufficient information for meta-analysis,42 ,43 ,47 ,52 ,65 ,71 resulting in a pooled sensitivity of 0.85 (95% CI 0.73 to 0.93) and specificity of 0.72 (95% CI 0.65 to 0.78) (figure 1A, B). This indicates that at this cut-point, the dermoscopy CPR is more useful for ruling out rather than ruling in melanoma, with a higher pooled sensitivity than specificity. I2 were high (>70%), indicating a high degree of heterogeneity. Of the seven studies excluded from meta-analysis, sensitivity ranged from 0.71 to 0.91 (mean 0.79) and specificity ranged from 0.43 to 0.92 (mean 0.72). None of the six studies that applied a cut-point of 5.45 were suitable for meta-analysis. From four studies that presented the information, sensitivity ranged from 0.73 to 0.98 (mean 0.85) and specificity ranged from 0.46 to 0.91 (mean 0.79) (table 4).

![Figure 1](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/7/3/e014096/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/7/3/e014096/F1)

Figure 1 
(A) Diagnostic accuracy ABCD rule with dermoscopy—pooled sensitivity and specificity (eight studies). (B) Summary receiver operating characteristic curves for ABCD rule of dermoscopy The circles represent individual studies and the size reflects the sample size. The red square represents the summary estimates of sensitivity and specificity and the dotted ellipses around this represent the 95% CI around the estimate. The 95% prediction region (amount of variation between studies) was wide, suggesting heterogeneity between studies. ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; HSROC, hierarchical summary receiver operating characteristic.

#### Seven-point checklist for dermoscopy

The 7-point checklist for dermoscopy was validated in 18 studies, 17 of which applied a cut-point of 3. 11 studies provided sufficient information for meta-analysis, revealing a pooled sensitivity of 0.77 (95% CI 0.61 to 0.88) and pooled specificity of 0.80 (95% CI 0.59 to 0.92) (See figure 2A, B).25–27 ,35 ,37 ,42 ,43 ,47 ,50 ,52 ,71 There was a high degree of heterogeneity in the results (I2>90%). Removing two outliers27 ,50 made minimal difference to the pooled result. Only one study validated the revised 7-point checklist for dermoscopy and reported sensitivity 0.78 and specificity 0.65 for a cut-point of 3 (table 4).27

![Figure 2](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/7/3/e014096/F2.medium.gif)

[Figure 2](http://bmjopen.bmj.com/content/7/3/e014096/F2)

Figure 2 
(A) Diagnostic accuracy of 7-point checklist with dermoscopy—pooled sensitivity and specificity (11 studies). (B) Summary receiver operating characteristic curves for ABCD rule of dermoscopy The circles represent individual studies and the size reflects the sample size. The red square represents the summary estimates of sensitivity and specificity and the dotted ellipses around this represent the 95% CI around the estimate. The 95% prediction region (amount of variation between studies) was wide, suggesting heterogeneity between studies. ABCD, Asymmetry, irregular Borders, more than one or uneven distribution of Colour, or a large (greater than 6 mm) Diameter; HSROC, hierarchical summary receiver operating characteristic; ROC, receiver operating characteristic.

### Impact analysis

We identified three unique studies that examined the impact of a melanoma CPR on processes of care (melanoma diagnosis and referrals), however, no patient outcomes were examined (table 2).62 ,63 The methodological quality of these studies is presented in online supplementary appendix 4.

Using a controlled before–after design, Westerhoff *et al* investigated the impact of an educational intervention about the Menzies 1996 rule on melanoma diagnosis by family physicians (FP). The control group did not receive the training. Postintervention, there was a significant improvement in melanoma diagnosis (75.9% vs 62.7%, p<0.001). No significant improvement was seen in the control group (54.8% vs 53.7%, p=0.59).62

Walter *et al* conducted a RCT to compare the use of a new imaging device, the MoleMate system (SIAscopy with a primary care scoring algorithm), to current best practice (clinical history, naked-eye examination, 7-point checklist). The authors found no difference between these two approaches in terms of appropriate referrals (the proportion of referred lesions that secondary care experts biopsied or monitored) to urgent skin cancer clinics (intervention 56.8% vs control 64.5% p=0.11) or the proportion of benign lesions appropriately managed in primary care (intervention 99.6% vs control 99.2%, p=0.46).63

Argenziano *et al*'s RCT,72 involved primary care physicians first attending a 1-day training course describing the ABCD rule (cut-point unspecified) and the 3-point checklist. They were then randomly assigned to assess patients with skin lesions, either by clinical (ie, naked eye) examination, or by dermoscopy using the 3-point checklist. The referral assessments were checked for accuracy by dermatologists. The dermoscopy arm demonstrated a 25% improvement in the sensitivity of primary care referrals of pigmented lesions compared with the naked-eye examination (79.2% vs 54.1%, p=0.002), without a reduction in specificity (71.8% vs 71.3%, p=0.915).72

## Discussion

### Summary of findings

This systematic review identified 48 studies validating a total of 24 CPRs for melanoma. Overall, the majority of validation studies used dermoscopic CPRs, with very few studies validating clinical CPRs. Meta-analysis of the dermoscopic CPRs demonstrated relatively high pooled estimates of sensitivity (0.77–0.86). The clinical implication is that applying dermoscopy CPRs will enable low-risk patients to be observed and kept under review in a primary care setting, without immediate referral for excision to secondary care. Meta-analysis was not possible for clinical CPRs but individual studies report variable sensitivity, ranging from 0.44 to 0.86. Three impact analysis studies were identified, with two reporting an improvement in melanoma diagnosis with the use of a CPR.

### Context of previous research

The sensitivities and specificities we report indicate that currently available CPRs are reasonably good at ruling out melanoma. The pooled sensitivity of the ABCD rule for dermoscopy (cut-point of >4.75) was 0.85 (95% CI 0.73 to 0.93), higher than that of the 7-point checklist for dermoscopy (0.77, 95% CI 0.61 to 0.88). While this evidence would support the use of such rules in prioritising appropriate referrals for higher risk patients and adopting a watchful waiting strategy in lower risk patients, there are a number of important caveats that may prevent their adoption in primary care.

Melanoma is a high-stakes condition, one which doctors tend to be cautious in diagnosing, often preferring to excise a benign lesion rather than to miss a potentially fatal cancer.73 In such cases, a CPR with near perfect sensitivity would be desirable, however, it has been argued that a lower sensitivity should not prevent CPR use unless usual decisions, made without the rule, are demonstrably better.74 Our results are comparable with previous systematic reviews focused on melanoma diagnosis across healthcare settings in highlighting that dermoscopic CPRs are demonstrably better in terms of diagnostic accuracy in comparison with inspection by the naked eye.16 ,75 However, even a rule with almost 100% sensitivity may not be adopted. For instance, implementation of the Canadian CT Head Rule, despite 100% sensitivity in validation studies, did not result in a reduction in imaging rates, with clinicians' reporting unease with certain components of the rule and fear of missing a high-stakes diagnosis as reasons for not adopting the CPR.76

Before considering whether to use a CPR in clinical practice, it is essential that its performance be established through external validation (ie, in settings outside where it was derived). We identified a number of external validation studies in this review, however, in keeping with much CPR research, the reporting of these studies was often poor.77 ,78 In particular, the common issues of limited acknowledgement and handling of missing data and key performance measures of prediction models, that is, calibration, being omitted was encountered.77 The lack of available data in some papers meant not all studies could be combined in the meta-analysis, meaning the sensitivities and specificities reported here are not based on the totality of existing evidence. Furthermore, we were unable to assess diagnostic accuracy at different cut-point thresholds for respective CPRs. Improved reporting of CPRs at cut-point thresholds will enable pooling of diagnostic accuracy data, and will provide more robust measures of diagnostic accuracy. After validation, impact analysis studies are undertaken to determine the impact of the implementation of a CPR on processes and outcomes of care. Despite increasing interest in developing and validating CPRs relevant to primary care, relatively few have undergone impact analysis.79 Despite the large number of CPRs identified in this review, we identified only three impact analysis studies, with only two studies reporting an improvement in correct melanoma diagnosis in primary care as a result. Arguably, the dearth of well conducted and clearly reported external validation and impact analysis studies undermines trust in the use of such rules in practice.77

Current NICE guidelines for melanoma detection and management recommend dermoscopy of any suspicious lesion, advising against using computer-assisted diagnostic tools (NG14) while promoting use of the weighted 7-point checklist in primary care to guide referral (NG12).20 Based on the findings of this review, the ABCD rule for dermoscopy had a higher sensitivity than the seven point for dermoscopy checklist at their respective cut-points, indicating its potential for use in primary care. Dermoscopy, however, requires training and equipment, and is less commonly performed in primary care. Evidence suggests that dermatologists have better diagnostic accuracy than primary care physicians.18 Three studies retrieved in our search assessed dermoscopy CPR performance when applied by non-experts, with two studies reporting that the CPRs performed well overall when used by non-experts, mainly primary care physicians.49 ,66 ,72 Westerhoff *et al*62 and Blum *et al*80 demonstrated that training primary care physicians to use dermoscopy with CPRs showed significant improvement in the diagnosis of melanoma compared with naked eye inspection. Alongside the use of CPRs, training in dermoscopy would seem to be a strategy that will enhance diagnostic accuracy of melanoma in the future particularly in light of emerging evidence of differences in dermoscopic features of melanoma such as head and neck melanoma.81 It has also been highlighted that significant efforts are needed to standardise and improve dermoscopic terminology to more broadly promote the use of dermoscopy in the primary care setting.82 Of the 24 rules identified in this review, four were clinical (ie, naked eye) and 17 were dermoscopic. Owing to the limited number of studies and available data, no meta-analysis of clinical CPRs could be conducted. The range of reported sensitivities from individual studies indicates that there is insufficient evidence to recommend their use in practice.

### Strengths and limitations of our study

The main strengths of this review are the use of broad inclusion criteria, the systematic search of multiple databases not limited by language, use of the CHARMS checklist to assess methodological quality, pooling data from a broad range of studies to enhance generalisability and the use of a broad definition of primary care to account for the variation in primary care services and access internationally. However, the findings of this systematic review need to be interpreted in the context of the limitations of the original studies. The lack of available data in some papers meant not all studies could be combined in the meta-analysis. A number of studies that validated CPRs and algorithms using novel diagnostic technologies which incorporated computerised image analysis and artificial intelligence were excluded from the review as routine use of these are not currently recommended in UK NICE clinical guidelines. Significant heterogeneity existed between the studies with respect to differences in the study populations and application of the CPR. Finally, individual patient data that enables pooling of risk scores at the different cut-points would enable researchers to explore the clinical use of applying risk scores at different cut-points with the purpose of assessing the role of melanoma CPRs at the different diagnostic thresholds of ‘ruling out’ (using highest pooled sensitivity) or ‘ruling in’ (using highest pooled specificity) of respective melanoma CPRs.

### Implications for practice and future research

Early detection followed by curative surgery greatly improves the prognosis of malignant melanoma. As the incidence of melanoma skin cancer increases, primary care physicians are increasingly required to screen for melanoma.12 Therefore, efforts to increase the early detection of melanoma must focus on supporting primary care physicians in performing skin cancer screenings with recent evidence highlighting the benefits of developing targeted screening strategies in high-risk patients in primary care.18 ,83 This systematic review identified 24 separate clinical (naked eye) and dermoscopic CPRs, with some overlap in the included the elements. Our analysis highlights that dermoscopic CPRs have reasonable sensitivity, with the ABCD rule for dermoscopy having better sensitivity than the 7-point checklist for dermoscopy. Further development of new rules is unlikely to benefit the field of research. An increased emphasis on better reporting of validation studies, particularly at different cut-point thresholds, would allow for the conduct of more robust diagnostic accuracy meta-analysis to inform decision making. Further methodologically robust RCTs are necessary also to examine the impact of implementing CPRs in clinical practice, in terms of patient outcomes, physician behaviour, processes of care and cost-effectiveness. Finally, while guidelines promote the use of dermoscopy in the assessment of pigmented skin lesions, there needs to be greater emphasis on training in primary care on this examination technique.

## Conclusion

This systematic review and meta-analysis shows that dermoscopic CPRs have reasonably high pooled estimates of sensitivity and may be a useful tool for primary care physicians prioritising appropriate referrals for higher risk patients and adopting a watchful waiting strategy in lower risk patients. The ABCD rule of dermoscopy has higher pooled sensitivity than the 7-point checklist for dermoscopy, when consideration about ruling out melanoma is being made. A focus on impact analysis may help translate melanoma CPRs into useful and effective triage tools for use in primary care.

## Footnotes

*   Contributors EH, NW and BC drafted the manuscript. EH, NW and BC contributed to development of the selection criteria, the risk of bias assessment strategy and the data extraction criteria. EH developed the search strategy. HB, LA and HS contributed the data extraction and quality assessments. BC and TF read, provided feedback and approved the final manuscript.

*   Funding This systematic review is funded by the HRB Centre for Primary Care Research under grant number HRC/2014/1, Royal College of Surgeons in Ireland (RCSI), Dublin, Ireland.

*   Competing interests None declared.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data sharing statement No additional data are available.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)

## References

1.  Erdmann F, Lortet-Tieulent J, Schüz J, et al. International trends in the incidence of malignant melanoma 1953–2008—are recent generations at higher or lower risk? Int J Cancer 2013;132: 385–400. [doi:10.1002/ijc.27616](http://dx.doi.org/10.1002/ijc.27616)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/ijc.27616&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22532371&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000311383600024&link_type=ISI) 

2.  Arnold M, Holterhues C, Hollestein LM, et al. Trends in incidence and predictions of cutaneous melanoma across Europe up to 2015. J Eur Acad Dermatol Venereol 2014;28:1170–8. [doi:10.1111/jdv.12236](http://dx.doi.org/10.1111/jdv.12236)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/jdv.12236&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23962170&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

3.  Coory M, Baade P, Aitken J, et al. Trends for in situ and invasive melanoma in Queensland, Australia, 1982–2002. Cancer Causes Control 2006;17:21–7. [doi:10.1007/s10552-005-3637-4](http://dx.doi.org/10.1007/s10552-005-3637-4)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1007/s10552-005-3637-4&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16411049&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000234825600004&link_type=ISI) 

4.  Jemal A, Saraiya M, Patel P, et al. Recent trends in cutaneous melanoma incidence and death rates in the United States, 1992–2006. J Am Acad Dermatol 2011;65(Suppl 1):S17–25.e11–13. [doi:10.1016/j.jaad.2011.04.032](http://dx.doi.org/10.1016/j.jaad.2011.04.032)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jaad.2011.04.032&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22018063&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

5.  de Vries E, Bray FI, Coebergh JW, et al. Changing epidemiology of malignant cutaneous melanoma in Europe 1953–1997: rising trends in incidence and mortality but recent stabilizations in western Europe and decreases in Scandinavia. Int J Cancer 2003;107:119–26. [doi:10.1002/ijc.11360](http://dx.doi.org/10.1002/ijc.11360)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/ijc.11360&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12925966&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000185244300017&link_type=ISI) 

6.  Thomas L, Tranchand P, Berard F, et al. Semiological value of ABCDE criteria in the diagnosis of cutaneous pigmented tumors. Dermatology (Basel) 1998;197:11–17. [doi:10.1159/000017969](http://dx.doi.org/10.1159/000017969)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1159/000017969&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=9693179&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000075079100003&link_type=ISI) 

7.  Cornish D, Holterhues C, van de Poll-Franse LV, et al. A systematic review of health-related quality of life in cutaneous melanoma. Annals Oncol 2009;20(Suppl 6):vi51–8. [doi:10.1093/annonc/mdp255](http://dx.doi.org/10.1093/annonc/mdp255)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiYW5ub25jIjtzOjU6InJlc2lkIjtzOjE1OiIyMC9zdXBwbF82L3ZpNTEiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNy8zL2UwMTQwOTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

8.  Guy GP, Machlin SR, Ekwueme DU, et al. Prevalence and costs of skin cancer treatment in the U.S., 2002–2006 and 2007–2011. Am J Prev Med 2015;48:183–7. [doi:10.1016/j.amepre.2014.08.036](http://dx.doi.org/10.1016/j.amepre.2014.08.036)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.amepre.2014.08.036&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=25442229&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

9.  Gandini S, Sera F, Cattaruzza MS, et al. Meta-analysis of risk factors for cutaneous melanoma: II. Sun exposure. Eur J Cancer 2005;41:45–60. [doi:10.1016/j.ejca.2004.10.016](http://dx.doi.org/10.1016/j.ejca.2004.10.016)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.ejca.2004.10.016&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15617990&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000226728700014&link_type=ISI) 

10. Gandini S, Sera F, Cattaruzza MS, et al. Meta-analysis of risk factors for cutaneous melanoma: I. Common and atypical naevi. Eur J Cancer 2005;41:28–44. [doi:10.1016/j.ejca.2004.10.015](http://dx.doi.org/10.1016/j.ejca.2004.10.015)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.ejca.2004.10.015&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15617989&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000226728700013&link_type=ISI) 

11. Gandini S, Sera F, Cattaruzza MS, et al. Meta-analysis of risk factors for cutaneous melanoma: III. Family history, actinic damage and phenotypic factors. Eur J Cancer 2005;41:2040–59. [doi:10.1016/j.ejca.2005.03.034](http://dx.doi.org/10.1016/j.ejca.2005.03.034)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.ejca.2005.03.034&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16125929&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000232648000011&link_type=ISI) 

12. Koelink CJL, Kollen BJ, Groenhof F, et al. Skin lesions suspected of malignancy: an increasing burden on general practice. BMC Fam Pract 2014;15:29. [doi:10.1186/1471-2296-15-29](http://dx.doi.org/10.1186/1471-2296-15-29)
    
    

13. Pockney P, Primrose J, George S, et al. Recognition of skin malignancy by general practitioners: observational study using data from a population-based randomised controlled trial. Br J Cancer 2009;100:24–7. [doi:10.1038/sj.bjc.6604810](http://dx.doi.org/10.1038/sj.bjc.6604810)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1038/sj.bjc.6604810&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19127264&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000262267700005&link_type=ISI) 

14. Argenziano G, Soyer HP, Chimenti S, et al. Dermoscopy of pigmented skin lesions: results of a consensus meeting via the internet. J Am Acad Dermatol 2003;48:679–93. [doi:10.1067/mjd.2003.281](http://dx.doi.org/10.1067/mjd.2003.281)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1067/mjd.2003.281&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12734496&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000182977200004&link_type=ISI) 

15. Benelli C, Roscetti E, Pozzo VD, et al. The dermoscopic versus the clinical diagnosis of melanoma. Eur J Dermatol 1999;9:470–6.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10491506&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000082889000010&link_type=ISI) 

16. Kittler H, Pehamberger H, Wolff K, et al. Diagnostic accuracy of dermoscopy. Lancet Oncol 2002;3:159–65. [doi:10.1016/S1470-2045(02)00679-4](http://dx.doi.org/10.1016/S1470-2045(02)00679-4)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S1470-2045(02)00679-4&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11902502&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000174890300020&link_type=ISI) 

17. Piccolo D, Ferrari A, Peris K, et al. Dermoscopic diagnosis by a trained clinician vs. a clinician with minimal dermoscopy training vs. computer-aided diagnosis of 341 pigmented skin lesions: a comparative study. Br J Dermatol 2002;147:481–6.
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2133.2002.04978.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12207587&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000177789600011&link_type=ISI) 

18. Chen SC, Pennie ML, Kolm P, et al. Diagnosing and managing cutaneous pigmented lesions: primary care physicians versus dermatologists. J Gen Intern Med 2006;21:678–82. [doi:10.1111/j.1525-1497.2006.00462.x](http://dx.doi.org/10.1111/j.1525-1497.2006.00462.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1525-1497.2006.00462.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16808765&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000238313000002&link_type=ISI) 

19. Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1–9. [doi:10.1186/2046-4053-4-1](http://dx.doi.org/10.1186/2046-4053-4-1)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/2046-4053-4-1&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=25554246&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

20. Macbeth F, Newton-Bishop J, O'Connell S, et al. Melanoma: summary of NICE guidance. BMJ 2015;351:h3708. doi: [10.1136/bmj.h3708](http://10.1136/bmj.h3708).
    
    [FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTEvanVsMjlfNi9oMzcwOCI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzMvZTAxNDA5Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

21. Moons KGM, de Groot JA, Bouwmeester W, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Medicine 2014;11:e1001744. [doi:10.1371/journal.pmed.1001744](http://dx.doi.org/10.1371/journal.pmed.1001744)
    
    

22. Higgins JPT, Green S eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. [http://handbook.cochrane.org](http://handbook.cochrane.org).
    
    

23. Reitsma JB, Glas AS, Rutjes AW, et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58:982–90. [doi:10.1016/j.jclinepi.2005.02.022](http://dx.doi.org/10.1016/j.jclinepi.2005.02.022)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2005.02.022&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16168343&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000232209000003&link_type=ISI) 

24. 1.  Deeks JJ, 
    2.  Bossuyt PM, 
    3.  Gatsonis C, eds
    
    Macaskill P, Gatsonis C, Deeks JJ, et al. Analysing and presenting results. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0. The Cochrane Collaboration, 2010. [http://methods.cochrane.org/sdt/handbook-dta-reviews/](http://methods.cochrane.org/sdt/handbook-dta-reviews/)
    
    

25. Annessi G, Bono R, Sampogna F, et al. Sensitivity, specificity, and diagnostic accuracy of three dermoscopic algorithmic methods in the diagnosis of doubtful melanocytic lesions. The importance of light brown structureless areas in differentiating atypical melanocytic nevi from thin melanomas. J Am Acad Dermatol 2007;56:759–67. [doi:10.1016/j.jaad.2007.01.014](http://dx.doi.org/10.1016/j.jaad.2007.01.014)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jaad.2007.01.014&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17316894&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000246041400005&link_type=ISI) 

26. Argenziano G, Fabbrocini G, Carli P, et al. Epiluminescence microscopy for the diagnosis of doubtful melanocytic skin lesions. Comparison of the ABCD rule of dermatoscopy and a new 7-point checklist based on pattern analysis. Arch Dermatol 1998;134:1563–70. [doi:10.1001/archderm.134.12.1563](http://dx.doi.org/10.1001/archderm.134.12.1563)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archderm.134.12.1563&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=9875194&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000077622100010&link_type=ISI) 

27. Argenziano G, Catricalà C, Ardigo M, et al. Seven-point checklist of dermoscopy revisited. Br J Dermatol 2011;164:785–90. [doi:10.1111/j.1365-2133.2010.10194.x](http://dx.doi.org/10.1111/j.1365-2133.2010.10194.x)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21175563&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

28. Benellii C, Roscetti E, Dal Pozzo V. The dermoscopic (7FFM) versus the clinical (ABCDE) diagnosis of small diameter melanoma. Eur J Dermatol 2000;10:282–7.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10846255&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

29. Carli P, Quercioli E, Sestini S, et al. Pattern analysis, not simplified algorithms, is the most reliable method for teaching dermoscopy for melanoma diagnosis to residents in dermatology. Br J Dermatol 2003;148:981–4. [doi:10.1046/j.1365-2133.2003.05023.x](http://dx.doi.org/10.1046/j.1365-2133.2003.05023.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2133.2003.05023.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12786829&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000183348200014&link_type=ISI) 

30. Dal Pozzo V, Benelli C, Roscetti E. The seven features for melanoma: a new dermoscopic algorithm for the diagnosis of malignant melanoma. Eur J Dermatol 1999;9:303–8.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10356410&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000080683400010&link_type=ISI) 

31. Piccolo D, Crisman G, Schoinas S, et al. Computer-automated abcd versus dermatologists with different degrees of experience in dermoscopy. Eur J Dermatol 2014;24:477–81. [doi:10.1684/ejd.2014.2320](http://dx.doi.org/10.1684/ejd.2014.2320)
    
    

32. Pizzichetta MA, Talamini R, Piccolo D, et al. Interobserver agreement of the dermoscopic diagnosis of 129 small melanocytic skin lesions. Tumori 2002;88:234–8.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12195762&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000177648800009&link_type=ISI) 

33. Soyer HP, Argenziano G, Zalaudek I, et al. Three-point checklist of dermoscopy. A new screening method for early detection of melanoma. Dermatology (Basel) 2004;208:27–31. [doi:10.1159/000075042](http://dx.doi.org/10.1159/000075042)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1159/000075042&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=14730233&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000188210300006&link_type=ISI) 

34. Strumia R, Montanari A. Low positive predictive value of ABCD-E rule for dermatoscopy of small melanocytic naevi. Melanoma Res 2003;13:631–2. [doi:10.1097/01.cmr.0000056281.15046.ea](http://dx.doi.org/10.1097/01.cmr.0000056281.15046.ea)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=14646628&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

35. Buhl T, Hansen-Hagge C, Korpas B, et al. Integrating static and dynamic features of melanoma: the DynaMel algorithm. J Am Acad Dermatol 2012;66:27–36. [doi:10.1016/j.jaad.2010.09.731](http://dx.doi.org/10.1016/j.jaad.2010.09.731)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21658795&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

36. Emery JD, Hunter J, Hall PN, et al. Accuracy of SIAscopy for pigmented skin lesions encountered in primary care: development and validation of a new diagnostic algorithm. BMC Dermatol 2010;10:9–9. [doi:10.1186/1471-5945-10-9](http://dx.doi.org/10.1186/1471-5945-10-9)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/1471-5945-10-9&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20868511&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

37. Haenssle HA, Korpas B, Hansen-Hagge C, et al. Seven-point checklist for dermatoscopy: performance during 10 years of prospective surveillance of patients at increased melanoma risk. J Am Acad Dermatol 2010;62:785–93. [doi:10.1016/j.jaad.2009.08.049](http://dx.doi.org/10.1016/j.jaad.2009.08.049)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20226567&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

38. Higgins EM, Hall P, Todd P, et al. The application of the seven-point check-list in the assessment of benign pigmented lesions. Clin Exp Dermatol 1992;17:313–5. [doi:10.1111/j.1365-2230.1992.tb00218.x](http://dx.doi.org/10.1111/j.1365-2230.1992.tb00218.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2230.1992.tb00218.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=1458634&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1992JN56600003&link_type=ISI) 

39. Kittler H, Seltenheim M, Dawid M, et al. Morphologic changes of pigmented skin lesions: a useful extension of the ABCD rule for dermatoscopy. J Am Acad Dermatol 1999;40:558–62. [doi:10.1016/S0190-9622(99)70437-8](http://dx.doi.org/10.1016/S0190-9622(99)70437-8)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0190-9622(99)70437-8&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10188673&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000079344100004&link_type=ISI) 

40. Keefe M, Dick DC, Wakeel RA. A study of the value of the seven-point checklist in distinguishing benign pigmented lesions from melanoma. Clin Exp Dermatol 1990;15:167–71. [doi:10.1111/j.1365-2230.1990.tb02064.x](http://dx.doi.org/10.1111/j.1365-2230.1990.tb02064.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2230.1990.tb02064.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=2142028&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1990DD04300003&link_type=ISI) 

41. McGovern TW, Litaker MS. Clinical predictors of malignant pigmented lesions. A comparison of the Glasgow seven-point checklist and the American Cancer Society's ABCDs of pigmented lesions. J Dermatol Surg Oncol 1992;18:22–6. [doi:10.1111/j.1524-4725.1992.tb03296.x](http://dx.doi.org/10.1111/j.1524-4725.1992.tb03296.x)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=1740563&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1992HC04300003&link_type=ISI) 

42. Skvara H, Teban L, Fiebiger M, et al. Limitations of dermoscopy in the recognition of melanoma. Arch Dermatol 2005;141:155–60. [doi:10.1001/archderm.141.2.155](http://dx.doi.org/10.1001/archderm.141.2.155)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archderm.141.2.155&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15724011&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000226913500005&link_type=ISI) 

43. Unlu E, Akay BN, Erdem C. Comparison of dermatoscopic diagnostic algorithms based on calculation: the ABCD rule of dermatoscopy, the seven-point checklist, the three-point checklist and the CASH algorithm in dermatoscopic evaluation of melanocytic lesions. J Dermatol 2014;41:598–603. [doi:10.1111/1346-8138.12491](http://dx.doi.org/10.1111/1346-8138.12491)
    
    

44. Walter FM, Prevost AT, Vasconcelos J, et al. Using the 7-point checklist as a diagnostic aid for pigmented skin lesions in general practice: a diagnostic validation study. Br J Gen Pract 2013;63:345–53. [doi:10.3399/bjgp13X667213](http://dx.doi.org/10.3399/bjgp13X667213)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.3399/bjgp13X667213&link_type=DOI) 

45. Osborne JE, Bourke JF, Graham-Brown RAC, et al. False negative clinical diagnoses of malignant melanoma. Br J Dermatol 1999;140:902–8. [doi:10.1046/j.1365-2133.1999.02823.x](http://dx.doi.org/10.1046/j.1365-2133.1999.02823.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2133.1999.02823.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10354030&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

46. Argenziano G, Longo C, Cameron A, et al. Blue-black rule: a simple dermoscopic clue to recognize pigmented nodular melanoma. Br J Dermatol 2011;165:1251–5. [doi:10.1111/j.1365-2133.2011.10621.x](http://dx.doi.org/10.1111/j.1365-2133.2011.10621.x)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21916885&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

47. Blum A, Rassner G, Garbe C. Modified ABC-point list of dermoscopy: a simplified and highly accurate dermoscopic algorithm for the diagnosis of cutaneous melanocytic lesions. J Am Acad Dermatol 2003;48:672–8. [doi:10.1067/mjd.2003.282](http://dx.doi.org/10.1067/mjd.2003.282)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1067/mjd.2003.282&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12734495&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000182977200003&link_type=ISI) 

48. Blum A, Luedtke H, Ellwanger U, et al. Digital image analysis for diagnosis of cutaneous melanoma. Development of a highly effective computer algorithm based on analysis of 837 melanocytic lesions. Br J Dermatol 2004;151:1029–38. [doi:10.1111/j.1365-2133.2004.06210.x](http://dx.doi.org/10.1111/j.1365-2133.2004.06210.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2133.2004.06210.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15541081&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000225080600010&link_type=ISI) 

49. Dolianitis C, Kelly J, Wolfe R, et al. Comparative performance of 4 dermoscopic algorithms by nonexperts for the diagnosis of melanocytic lesions. Arch Dermatol 2005;141:1008–14. [doi:10.1001/archderm.141.8.1008](http://dx.doi.org/10.1001/archderm.141.8.1008)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archderm.141.8.1008&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16103330&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000231072100010&link_type=ISI) 

50. Gereli MC, Onsun N, Atilganoglu U, et al. Comparison of two dermoscopic techniques in the diagnosis of clinically atypical pigmented skin lesions and melanoma: seven-point and three-point checklists. Int J Dermatol 2010;49:33–8. [doi:10.1111/j.1365-4632.2009.04152.x](http://dx.doi.org/10.1111/j.1365-4632.2009.04152.x)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20465608&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

51. Guitera P, Menzies SW, Longo C, et al. In vivo confocal microscopy for diagnosis of melanoma and basal cell carcinoma using a 2-step method: analysis of 710 consecutive clinically equivocal cases. Australas J Dermatol 2012;53:7–8.
    
    

52. Henning JS, Stein JA, Yeung J, et al. CASH algorithm for dermoscopy revisited. Arch Dermatol 2008;144:554–5. [doi:10.1001/archderm.144.4.554](http://dx.doi.org/10.1001/archderm.144.4.554)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archderm.144.4.554&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18427058&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

53. Lorentzen H, Weismann K, Kenet RO, et al. Comparison of dermatoscopic ABCD rule and risk stratification in the diagnosis of malignant melanoma. Acta Derm Venereol 2000;80:122–6.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10877133&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

54. Luttrell MJ, McClenahan P, Hofmann-Wellenhof R, et al. Laypersons’ sensitivity for melanoma identification is higher with dermoscopy images than clinical photographs. Br J Dermatol 2012;167:1037–41. [doi:10.1111/j.1365-2133.2012.11130.x](http://dx.doi.org/10.1111/j.1365-2133.2012.11130.x)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22762457&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

55. MacKie RM, Fleming C, McMahon AD, et al. The use of the dermatoscope to identify early melanoma using the three-colour test. Br J Dermatol 2002;146:481–4. [doi:10.1046/j.1365-2133.2002.04587.x](http://dx.doi.org/10.1046/j.1365-2133.2002.04587.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2133.2002.04587.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11952549&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

56. Menzies SW, Kreusch J, Byth K, et al. Dermoscopic evaluation of amelanotic and hypomelanotic melanoma. Arch Dermatol 2008;144:1120–7. [doi:10.1001/archderm.144.9.1120](http://dx.doi.org/10.1001/archderm.144.9.1120)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archderm.144.9.1120&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18794455&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

57. Menzies SW, Moloney FJ, Byth K, et al. Dermoscopic evaluation of nodular melanoma. JAMA Dermatol 2013;149:699–709. [doi:10.1001/jamadermatol.2013.2466](http://dx.doi.org/10.1001/jamadermatol.2013.2466)
    
    

58. Argenziano G, Soyer HP, Chimenti S, et al. Impact of dermoscopy on the clinical management of pigmented skin lesions. Clin Dermatol 2002;20:200–2. [doi:10.1016/S0738-081X(02)00234-1](http://dx.doi.org/10.1016/S0738-081X(02)00234-1)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=12074853&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

59. Wadhawan T, Situ N, Rui H, et al. Implementation of the 7-point checklist for melanoma detection on smart handheld devices. Conf Proc IEEE Eng Med Biol Soc 2011;2011:3180–3.
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22255015&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

60. Zalaudek I, Argenziano G, Soyer HP, et al. Three-point checklist of dermoscopy: an open internet study. Br J Dermatol 2006;154:431–7. [doi:10.1111/j.1365-2133.2005.06983.x](http://dx.doi.org/10.1111/j.1365-2133.2005.06983.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2133.2005.06983.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16445771&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000234918900007&link_type=ISI) 

61. Walter FM, Prevost AT, Vasconcelos J, et al. The diagnostic accuracy of the 7-point checklist to assess pigmented skin lesions in primary care. Asia Pac J Clin Oncol 2012;8:311.
    
    

62. Westerhoff K, McCarthy WH, Menzies SW. Increase in the sensitivity for melanoma diagnosis by primary care physicians using skin surface microscopy. Br J Dermatol 2000;143:1016–20. [doi:10.1046/j.1365-2133.2000.03836.x](http://dx.doi.org/10.1046/j.1365-2133.2000.03836.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1046/j.1365-2133.2000.03836.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=11069512&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000165397700018&link_type=ISI) 

63. Walter FM, Morris HC, Humphrys E, et al. Effect of adding a diagnostic aid to best practice to manage suspicious pigmented lesions in primary care: randomised controlled trial. BMJ (Online) 2012;345:e4110. [doi:10.1136/bmj.e4110](http://dx.doi.org/10.1136/bmj.e4110)
    
    

64. Healsmith MF, Bourke JF, Osborne JE, et al. An evaluation of the revised seven-point checklist for the early diagnosis of cutaneous malignant melanoma. Br J Dermatol 1994;130:48–50. [doi:10.1111/j.1365-2133.1994.tb06881.x](http://dx.doi.org/10.1111/j.1365-2133.1994.tb06881.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2133.1994.tb06881.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=8305316&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1994MQ29000009&link_type=ISI) 

65. Rao BK, Marghoob AA, Stolz W, et al. Can early malignant melanoma be differentiated from atypical melanocytic nevi by in vivo techniques? Part I. Clinical and dermoscopic characteristics. Skin Res Technol 1997;3:8–14. [doi:10.1111/j.1600-0846.1997.tb00153.x](http://dx.doi.org/10.1111/j.1600-0846.1997.tb00153.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1600-0846.1997.tb00153.x&link_type=DOI) 

66. Binder M, Kittler H, Steiner A, et al. Reevaluation of the ABCD rule for epiluminescence microscopy. J Am Acad Dermatol 1999;40I):171–6. [doi:10.1016/S0190-9622(99)70184-2](http://dx.doi.org/10.1016/S0190-9622(99)70184-2)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0190-9622(99)70184-2&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10025741&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000078428000004&link_type=ISI) 

67. Feldmann R, Fellenz C, Gschnait F. The ABCD rule in dermatoscopy: analysis of 500 melanocytic lesions. Hautarzt 1998;49:473–6. [doi:10.1007/s001050050772](http://dx.doi.org/10.1007/s001050050772)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=9675574&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

68. Lorentzen H, Weismann K, Secher L, et al. The dermatoscopic ABCD rule does not improve diagnostic accuracy of malignant melanoma. Acta Derm Venereol 1999;79:469–72. [doi:10.1080/000155599750009942](http://dx.doi.org/10.1080/000155599750009942)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10598764&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

69. Nachbar F, Stolz W, Merkle T, et al. The ABCD rule of dermatoscopy. High prospective value in the diagnosis of doubtful melanocytic skin lesions. J Am Acad Dermatol 1994;30:551–9. [doi:10.1016/S0190-9622(94)70061-3](http://dx.doi.org/10.1016/S0190-9622(94)70061-3)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0190-9622(94)70061-3&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=8157780&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1994NE85200004&link_type=ISI) 

70. Stolz W, Riemann A, Cognetta AB, et al. ABCD rule of dermatoscopy: a new practical method for early recognition of malignant melanoma. Eur J Dermatol 1994;4:521–7.
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1994PQ37200005&link_type=ISI) 

71. Blum A, Clemens J, Argenziano G. Three-colour test in dermoscopy: a re-evaluation. Br J Dermatol 2004;150:1040. [doi:10.1111/j.1365-2133.2004.05941.x](http://dx.doi.org/10.1111/j.1365-2133.2004.05941.x)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15149530&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

72. Argenziano G, Puig S, Zalaudek I, et al. Dermoscopy improves accuracy of primary care physicians to triage lesions Suggestive of Skin Cancer. J Clin Oncol 2006;24:1877–82. [doi:10.1200/JCO.2005.05.0864](http://dx.doi.org/10.1200/JCO.2005.05.0864)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiamNvIjtzOjU6InJlc2lkIjtzOjEwOiIyNC8xMi8xODc3IjtzOjQ6ImF0b20iO3M6MjU6Ii9ibWpvcGVuLzcvMy9lMDE0MDk2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

73. Carli P, Nardini P, Crocetti E, et al. Frequency and characteristics of melanomas missed at a pigmented lesion clinic: a registry-based study. Melanoma Res 2004;14:403–7. [doi:10.1097/00008390-200410000-00011](http://dx.doi.org/10.1097/00008390-200410000-00011)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/00008390-200410000-00011&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15457097&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000224649600011&link_type=ISI) 

74. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med 2006;144:201–9. [doi:10.7326/0003-4819-144-3-200602070-00009](http://dx.doi.org/10.7326/0003-4819-144-3-200602070-00009)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.7326/0003-4819-144-3-200602070-00009&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16461965&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000235074800007&link_type=ISI) 

75. Rajpara SM, Botello AP, Townend J, et al. Systematic review of dermoscopy and digital dermoscopy/ artificial intelligence for the diagnosis of melanoma. Br J Dermatol 2009;161:591–604. [doi:10.1111/j.1365-2133.2009.09093.x](http://dx.doi.org/10.1111/j.1365-2133.2009.09093.x)
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19302072&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000269181000016&link_type=ISI) 

76. Stiell IG, Clement CM, Grimshaw JM, et al. A prospective cluster-randomized trial to implement the Canadian CT Head Rule in emergency departments. CMAJ 2010;182:1527–32. [doi:10.1503/cmaj.091974](http://dx.doi.org/10.1503/cmaj.091974)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTgyLzE0LzE1MjciO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNy8zL2UwMTQwOTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

77. Collins GS, de Groot JA, Dutton S, et al. External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med Res Methodol 2014;14:40. [doi:10.1186/1471-2288-14-40.](http://dx.doi.org/10.1186/1471-2288-14-40.)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/1471-2288-14-40&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=24645774&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 

78. Riley RD, Ensor J, Snell KIE, et al. External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ 2016;353:i3140. [doi:10.1136/bmj.i3140](http://dx.doi.org/10.1136/bmj.i3140)
    
    [FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTMvanVuMjJfNy9pMzE0MCI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzMvZTAxNDA5Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

79. Wallace E, Uijen MJM, Clyne B, et al. Impact analysis studies of clinical prediction rules relevant to primary care: a systematic review. BMJ Open 2016;6. [doi:10.1136/bmjopen-2015-009957](http://dx.doi.org/10.1136/bmjopen-2015-009957)
    
    

80. Blum A, Clemens J, Argenziano G. Modified dermoscopic algorithm for the differentiation between melanocytic and nonmelanocytic skin tumors. J Cutan Med Surg 2006;10:73–8. [doi:10.2310/7750.2006.00021](http://dx.doi.org/10.2310/7750.2006.00021)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BjbXMiO3M6NToicmVzaWQiO3M6NzoiMTAvMi83MyI7czo0OiJhdG9tIjtzOjI1OiIvYm1qb3Blbi83LzMvZTAxNDA5Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

81. Cengiz FP, Cengiz AB, Emiroglu N, et al. Dermoscopic and clinical features of head and neck melanoma. An Bras Dermatol 2015;90:488–93. [doi:10.1590/abd1806-4841.20153341](http://dx.doi.org/10.1590/abd1806-4841.20153341)
    
    

82. Carrera C, Marchetti MA, Dusza SW, et al. Validity and reliability of dermoscopic criteria used to differentiate nevi from melanoma: a web-based international dermoscopy society study. JAMA Dermatol 2016;152:798–806. [doi:10.1001/jamadermatol.2016.0624](http://dx.doi.org/10.1001/jamadermatol.2016.0624)
    
    

83. Rat C, Grimault C, Quereux G, et al. Proposal for an annual skin examination by a general practitioner for patients at high risk for melanoma: a French cohort study. BMJ OPEN 2015;5:e007471. [doi:10.1136/bmjopen-2014-007471](http://dx.doi.org/10.1136/bmjopen-2014-007471)
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiNS83L2UwMDc0NzEiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vNy8zL2UwMTQwOTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

84. Kreusch J, Rassner G, Trahn C, et al. Epiluminescent microscopy: a score of morphological features to identify malignant melanoma. Pigment Cell Res 1992;(Suppl 2):295–8.
    
    

85. Menzies SW, Ingvar C, Crotty KA, et al. Frequency and morphologic characteristics of invasive melanomas lacking specific surface microscopic features. Arch Dermatol 1996;132:1178–82. [doi:10.1001/archderm.1996.03890340038007](http://dx.doi.org/10.1001/archderm.1996.03890340038007)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archderm.1996.03890340038007&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=8859028&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1996VM03900006&link_type=ISI) 

86. Nilles M, Boedeker RH, Schill WB. Surface microscopy of naevi and melanomas—clues to melanoma. Br J Dermatol 1994;130:349–55. [doi:10.1111/j.1365-2133.1994.tb02932.x](http://dx.doi.org/10.1111/j.1365-2133.1994.tb02932.x)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2133.1994.tb02932.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=8148277&link_type=MED&atom=%2Fbmjopen%2F7%2F3%2Fe014096.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1994MZ17600013&link_type=ISI)