Article Text

Protocol
Assessment of the methodological, recommendation and reporting quality of global guidelines for neck pain and synthesis of evidence and recommendations: a systematic review protocol
  1. Jiayu Li1,2,
  2. Jiayin Ou2,
  3. Yang Liu3,
  4. Chenwei Shen1,
  5. Xiaoli Chen4,
  6. Ying Li5,
  7. Jiayi Zhao1,
  8. Jing Xu1,
  9. Yu Zhang6,
  10. Lin Wang1
  1. 1South China Research Center for Acupuncture and Moxibustion, Guangzhou University of Chinese Medicine, Guangzhou, China
  2. 2Science and Technology Innovation Center, Guangzhou University of Chinese Medicine, Guangzhou, China
  3. 3The Second Clinical Medical College, Guangzhou University of Chinese Medicine, Guangzhou, China
  4. 4Guangzhou Huanan Business College, Guangzhou, China
  5. 5Clinical Medical College of Acupuncture Moxibustion and Rehabilitation, Guangzhou University of Chinese Medicine, Guangzhou, China
  6. 6The First Affiliated Hospital of Anhui University of Traditional Chinese Medicine, Hefei, Anhui, China
  1. Correspondence to Dr Yu Zhang; zhangyudyx92{at}163.com; Dr Lin Wang; wanglin16{at}gzucm.edu.cn

Abstract

Introduction Neck pain is a global health problem that can cause severe disability and a huge medical burden. Clinical practice guideline (CPG) is an important basis for clinical diagnosis and treatment. A high-quality CPG plays a significant role in clinical practice. However, the quality of the CPGs for neck pain lacks comprehensive assessment. This protocol aims to evaluate the methodological, recommendation, reporting quality of global CPGs for neck pain and identify key recommendations and gaps that limit evidence-based practice.

Method CPGs from January 2013 to November 2023 will be identified through a systematic search on 13 scientific databases (PubMed, Cochrane Library, Embase, etc) and 7 online guideline repositories. Six reviewers will independently evaluate the quality of CPGs for neck pain by using the Appraisal of Guidelines for Research and Evaluation, the Appraisal of Guidelines Research and Evaluation-Recommendations Excellence and the Reporting Items for Practice Guidelines in Healthcare tools. Intraclass correlation coefficient will be used to test the consistency of the assessment. We will identify the distribution of evidence and recommendations in each evidence-based CPGs for neck pain and regrade the level of evidence and strength of recommendations by adopting the commonly used Grading of Recommendations, Assessment, Development and Evaluations system. The key recommendations based on high-quality evidence will be summarised. In addition, we will categorise CPGs by different characteristics and conduct a subgroup analysis of the results of assessment.

Ethics and dissemination No subjects will be involved in this systematic review, so there is no need for ethical approval. The finding of this review will be summarised as a paper for publication in a peer-reviewed journal.

PROSPERO registration number CRD42023417717.

  • bone diseases
  • protocols & guidelines
  • rehabilitation medicine
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • This review will evaluate the quality of clinical practice guidelines (CPGs) for neck pain from multiple dimensions.

  • This review will regrade the level of evidence and strength of recommendations of all included evidenced-based CPGs for neck pain.

  • This review will explore the factors that influence the quality of CPGs for neck pain.

  • This is a broad systematic review protocol that will be conducted without language restrictions on the inclusion of CPGs.

  • Although reviewers have received training about the use of the quality evaluation tools, it is inevitable that the scoring results will be affected to subjectivity during the actual assessment process.

Introduction

Neck pain is usually classified as traumatic (whiplash-associated disorders) and non-traumatic neck pain.1 Studies have shown that about two-thirds of people have neck pain at some time in their lives,2 and about 1.7%–11.5% of patients have limited mobility due to neck pain.3 A report in The Lancet in 2017 pointed out that the loss of healthy lifespan from neck pain increased by 21.9% from 2006 to 2016.4 Neck pain has become the leading cause of disability worldwide, resulting in a huge medical burden and social labour loss.5

Clinical practice guideline (CPG) was first defined in 1990 by the Institute of Medicine as ‘systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances’,6 and was updated in 2011 as ‘statements that include recommendations intended to optimise patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options’.7 CPG provides medical staff with guidance on medical practice, and is an important basis for clinical diagnosis and treatment of patients. In addition, CPG plays a significant role in health policy development, including multiple aspects of healthcare, such as health promotion and disease screening.8 As a programmatic document guiding clinical practice, the quality of CPGs is particularly crucial, but currently more than half of them are considered unreliable.9 In 2018, JAMA published a paper pointing out that despite great progress in the development of CPGs, serious problems remain.10

Various tools have been developed to assess the quality of CPGs.11 Among them, the Appraisal of Guidelines for Research and Evaluation II (AGREE II) is currently the preferred tool for quality assessment of CPGs worldwide, which can be used to assess the methodological quality of CPGs and provide methodological strategies for the development of new CPGs.12 However, while a CPG gets a high AGREE II score, there is no guarantee that the resulting CPG recommendations are optimal.13–17 CPG developers may not take common clinical conditions into consideration when forming recommendations.16 Therefore, as a complementary tool to AGREE II, the Appraisal of Guidelines Research and Evaluation-Recommendations Excellence (AGREE-REX) tool can be applied to evaluate the clinical credibility and implementation of recommendations.18 And the Reporting Items for Practice Guidelines in Healthcare (RIGHT) checklist was developed to evaluate the reporting quality of CPGs.19 In addition, evidence is a core component of evidence-based medicine and practice.20 The evaluation of published evidence also plays an important part in evidence-based practice.21 The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system22 23 has been recognised as the most ideal and commonly used system for grading evidence and recommendations.

Currently, published systematic reviews24–26 related to neck pain guidelines mainly focused on the methodological quality of CPGs for neck pain. However, no attention was paid to the quality of recommendations and reporting. Research evidence, values and local/regional circumstances should all be considered in the CPGs,27 as well as the transparency and standardisation of reporting.19 However, AGREE II items are inadequate in these domains. In addition, the CPGs included in these studies were limited in terms of target population/language/region. The analysis and synthesis of the level of evidence and strength of recommendations were not reported. There was also a lack of in-depth exploration of the relationship between the characteristics of CPGs and the quality of CPGs. Furthermore, the inclusion and exclusion criteria employed in these reviews exhibit imperfections, and there has been a development of updated CPGs in recent years. Therefore, a comprehensive and broad systematic review is necessary. This protocol aims to evaluate the quality of global CPGs for neck pain (with no language restrictions for inclusion) in multiple dimensions, identify the distribution of evidence and recommendations in each evidence-based CPGs, recognise potential factors that influence the development of CPG and summarise key recommendations that were based on high-quality evidence across each CPGs.

Methods and analysis

Protocol and registration

This systematic review protocol follows the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P)28 statement (see online supplemental material 1). And it has been registered in the International Prospective Register of Systematic Reviews (PROSPERO) with the registration number CRD42023417717.

Patient and public involvement

No patients or/and the public will be involved in the development of this protocol.

Search strategy

For a broad collection of CPGs for neck pain, we referred to high-quality systematic reviews29–31 of CPGs without language restrictions for inclusion. And all the databases mentioned in these reviews were collected for retrieval. We will conduct a comprehensive search in the following databases/repositories:

  1. Thirteen scientific databases: PubMed, the Cochrane Library, Embase, Physiotherapy Evidence Database (PEDro), Scopus, Web of Science, Epistemonikos, Tripdatabase, MEDLINE, Cumulative Index to Nursing and Allied Health Literature, China National Knowledge Infrastructure, WanFang Data and Chinese Scientific Journal Database.

  2. Seven online guideline repositories: the National Institute for Health and Clinical Excellence, the Scottish Intercollegiate Guidelines Network, the National Health and Medical Research Council, WHO and Guidelines International Networks, Agency for Healthcare Research and Quality.

  3. In order to identify potential eligible CPGs, we will also search the reference list of included CPGs. For CPGs that were not written in English, we will search for the translations or contact the CPG developers via the email to provide the corresponding translation.

Terms for neck pain and CPGs will be adopted for the search in each database. The search strategy will be adjusted according to the database consulted. The search range will be from January 2013 to November 2023 and the language of CPGs will be unrestricted. Detailed search strategies were listed in online supplemental material 2.

Study selection

CPGs will be included if they: (1) focused on diagnosis and management of neck pain or acute and chronic whiplash-associated disorder grades I–III or non-traumatic neck pain; (2) were published between January 2013 and November 2023; (3) met the definition of CPG. Both the evidence-based CPGs and consensus-based CPGs will be included. No language restrictions will be placed on the CPGs for neck pain.

CPGs will be excluded if: (1) full texts were unavailable; (2) they were editorials, comments, reviews and correspondence studies; (3) they were the interpretation and adaptation of CPGs; (4) they are the previous version of the updated CPG; (5) focused on legal issues, epidemiological investigation, training and research methods.

The selection process will be presented in a flow chart in accordance with PRISMA (figure 1). The retrieved publications will be exported into EndNote X7 literature management software (Thomson Reuters, Stanford, USA), and duplicates will be excluded. Two reviewers (CS and XC) will first review the titles and abstracts of the studies for screening to exclude the studies that do not meet the eligibility criteria, and then read the full texts of the remaining studies to determine if they will be finally included. Any disagreements will be resolved through discussion or consultation with the corresponding authors.

Figure 1

PRISMA flow diagram. AHRQ, Agency for Healthcare Research and Quality; CINAHL, Cumulative Index to Nursing and Allied Health Literature; CNKI, China National Knowledge Infrastructure; GIN, Guidelines International Networks; NHMRC, the National Health and Medical Research Council; NICE, the National Institute for Health and Clinical Excellence; PEDro, Physiotherapy Evidence Database; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; SIGN, the Scottish Intercollegiate Guidelines Network; VIP, Chinese Scientific Journal Database; WHO, World Health Organization.

Data extraction

Data extraction will be performed by three reviewers (JZ, JX and JL). In order to minimise the omission of important potential information, we will collect and analyse all documents related to the included CPGs. Two reviewers (JZ and JX) will form a group to extract the data of the CPGs on neck pain, and will record the relevant information according to the data extraction list. The consistency of extracted data will be checked by the third reviewer (JL).

General characteristics

The following general characteristics of CPGs for neck pain will be extracted: ID (the first author, year of publication), country of the first author, organisation responsible for CPG, target group, scope, funding sources, version, grading system, development method. Then we will classify the following characteristics as: year of publication (>2018/≤2018), country of the first author (low-income and middle-income country/high-income country), version (first/updated), development method (evidence-based/consensus-based), scope (diagnosis/treatment/prevention/others); methodologists included (yes/no), used CPG quality tool (yes/no), language (English/Chinese/others).

Grading system, recommendations and evidence

We will record the grading systems of recommendations and evidence applied in each evidence-based CPG and collect the rating standards for these systems. In addition, all the recommendations about the management of neck pain will be extracted from CPGs for neck pain. And the strength of recommendations and the corresponding level of evidence will also be recorded. When a recommendation was based on multiple levels of evidence, we will record only the highest level of evidence. When the strength of recommendation or the level of evidence was not explicitly stated in the CPGs, we will record ‘not applicable’.

Quality assessment

Six reviewers (JL, JO, YaL, CS, YiL and XC) will independently evaluate the quality of CPGs for neck pain by using AGREE II, AGREE-REX, RIGHT tools. And before the assessment, the reviewers will read the user manual and receive online training concerning these three tools. When significant differences of opinion emerge, they will discuss with the corresponding author. Intraclass correlation coefficient (ICC) will be used to detect the consistency to ensure that six reviewers’ understanding of each item is basically the same, which will help to improve the objectivity and reliability of evaluations. Formal assessment will be performed only when ICC >0.8.32

AGREE II12 will be used to assess the methodological quality of CPGs, and the latest version of AGREE II updated in 2017 will be used in this study. AGREE II contains 23 items categorised into 6 domains: domain 1 (scope and purpose), domain 2 (stakeholder involvement), domain 3 (rigour of development), domain 4 (clarity of presentation), domain 5 (applicability) and domain 6 (editorial independence). AGREE-REX18 will be applied to evaluate the quality of CPGs’ recommendations. AGREE-REX includes nine key items organised in three domains: domain 1 (clinical applicability), domain 2 (values and preferences) and domain 3 (implementability). Both AGREE II and AGREE-REX quality evaluation items are rated on a 7-point scale, with 1 for strongly disagree and 7 for strongly agree. The standardised scores in each domain are calculated according to the following formula: AGREE II domain score=(obtained score–minimum possible score)/(maximum possible score–minimum possible score)×100%, AGREE-REX domain score=(consensus score–lowest possible score)/(highest possible score–lowest possible score)×100%.12 A higher score means the guide performs better in the relevant domain. Because the method for evaluating the overall quality of CPGs remains controversial,33 we will only present the results of the reviewers’ subjective judgements.

We will use RIGHT19 checklist to assess the reporting quality of CPGs for neck pain. The RIGHT checklist includes 35 items in 7 domains: domain 1 (basic information), domain 2 (background), domain 3 (evidence), domain 4 (recommendations), domain 5 (review and quality assurance), domain 6 (funding and declaration and management of interest) and domain 7 (other information). Based on whether the CPG provides relevant information, each item well be classified into two grades: ‘reported’ and ‘not reported’. We calculated each domain score and overall score according to the following formula34: RIGHT domain score=total number of items ‘reported’ in each domain/total number of items in each domain×100%; RIGHT overall score=total number of items ‘reported’ in each domain/35×100%. Generally, score of >80 will be considered ‘well-reported’, score of 50–80 will be considered ‘moderate-reported’ and score of <50 will be considered ‘low-reported’.

Level of evidence and strength of recommendations

Despite the evidence grading systems adopted by the evidence-based CPGs for neck pain may be varied, most of these systems have similar and comparable structures. There are some studies of CPG quality assessment that represent recommendations and evidence by using a uniform criterion.35–37 Therefore, in order to present an overall condition of the level of evidence and strength of recommendations among all CPGs for neck pain, all CPG evidence grading systems for neck pain were standardised into the GRADE system by two reviewers (JO and YaL). Based on this classification system, the level of evidence and level of recommendation of each CPG were recorded. Before the regrading process, we will interpret the grading standards of each grading systems mentioned in different CPGs for neck pain, and then correspond the different grades of the system to the structure of GRADE system. The regrading of recommendations and evidence will be mainly based on the strength and level mentioned in the neck pain CPGs, and reviewers could standardise each recommendation and evidence into the GRADE system in combination with other considerations beyond the level of evidence body. The evidence will be divided into four levels: ‘high’, ‘moderate’, ‘low’ and ‘very low’, and the strength of recommendations will be divided into two grades: ‘strong’ and ‘weak’. And when there is no evidence for certain recommendation, which is based solely on consensus and expert opinion, the evidence level will be classified as ‘very low’. All disputes in the regrade process were resolved by discussion with the corresponding author.

Key recommendations in CPGs for neck pain

After recommendations were extracted in CPGs for neck pain, two reviewers (JL and YaL) will independently perform a synthesis of recommendations. The process will be as follows: (1) classify the types of recommendations (diagnosis, treatment, rehabilitation, prevention, etc); (2) integrate recommendations that are same or similar among CPGs for neck pain; (3) identify conflicting recommendations and trace their sources of evidence; (4) highlight recommendations based on high-quality of evidence and (5) list recommendations with consistent strength across all CPGs. The third reviewer (JO) will compare the results of the recommendation synthesis and determine the final list of key recommendations.

Statistical analysis

We will analyse all data by SPSS V.28.0 (IBM, Illinois, USA) software, R V.3.4.3 (http://www.R-project.org; The R Foundation), EmpowerStats V.4.1 (http://www.empowerstats.com; X&Y Solutions, Massachusetts, USA).

The standardised score for each domain and over score of each CPG on neck pain will be expressed as mean±SD. And the distribution between the level of evidence and strength of recommendation will be expressed as frequency, percentage, mean (SD) and median (Q1–Q3). ICC with 95% CI will be used to test for agreement among the six researchers and assess inter-rater reliability. Generally, an ICC of <0.40 was classified as poor, an ICC of 0.40–0.59 was classified as fair, an ICC of 0.60–0.75 was classified as good and an ICC of >0.75 was classified as excellent.

In addition, we will conduct a subgroup analysis of AGREE II, AGREE-REX and RIGHT score. The extracted classification variables will be expressed as the number of case and frequency. The independent-sample t-test/analysis of variance/Kruskal-Wallis (H) test will be used to explore the differences between different groups. We will use Spearman’s correlation to explore the association among the AGREE II, AGREE-REX and RIGHT domains.

Discussion

With the development of society and the changes in people’s lifestyle, the incidence of neck pain is also steadily increasing. The clinical efficacy of treatments related to neck pain remain poor and the clinical status of neck pain management needs further improvement. CPGs are fundamental for evidence-based medicine.38 But increasely studies about the evaluation of CPG quality have reported that the overall quality is poor. Hence, a comprehensive quality study of CPG for neck pain is essential.

This protocol provides a standardised method and process for assessing the quality of global CPGs for neck pain. The methodology, recommendation, reporting, evidence quality of the CPGs for neck pain will be reviewed. We will also conduct an extended analysis of the factors affecting CPGs’ quality, the association of AGREE II, AGREE-REX, RIGHT domains. By adopting GRADE system as the uniform criteria, the distribution of the level of evidence and strength of recommendations will be clearly identified. In addition, we will conclude a key recommendation list for neck pain. A comprehensive evaluation of the quality of CPGs for neck pain can help CPG developers identify and improve the shortcomings of CPGs, help doctors better apply CPGs and improve the safety of medical services, and help patients obtain better treatment. Our protocol may have some shortcomings. First, study selection may be affected by selection bias. Although this protocol involves broad retrieval, it is inevitable that some literature may be omitted, which may affect the understanding of overall CPGs quality. To alleviate this limitation, we will state the criteria for study selection and provide specific reasons for exclusion in the process of study selection. Second, when extracting data, it may be affected by information bias in literature reports. Some CPGs may not fully report the data. We will provide sufficient transparency of detailed steps of data extraction. Third, although reviewers have received training about the use of the quality evaluation tools, it is inevitable that the scoring results will be affected to subjectivity during the actual assessment process. ICC will be used to measure the consistency of scoring.

Overall, the main purpose of this protocol is to determine the current quality of the CPGs for neck pain and identify the existing problems of the CPGs for neck pain, so as to provide effective suggestions for CPG developers and improve the clinical management of neck pain.

Ethics and dissemination

No subjects will be involved in this systematic review, so there is no need for ethical approval. The finding of this review will be summarised as a paper for publication in a peer-reviewed journal.

Ethics statements

Patient consent for publication

References

View Abstract

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • JL and JO are joint first authors.

  • Contributors JL and JO conceived this study. CS and XC designed the search strategies. JL, YaL and JZ designed the appraisal strategy of each included CPGs. JL, YaL and CS designed the synthesis of evidence and recommendations. JO, JL, JX and YiL drafted the protocol. LW and YZ rigorously revised the manuscript and made substantial intellectual contribution. JL and JO contributed equally to this paper. All authors read and agreed the final manuscript.

  • Funding This study was supported by the Youth Programme of the National Natural Science Foundation of China (No: 81904297); the Innovation Team and Talents Cultivation Programme of National Administration of Traditional Chinese Medicine (No: ZYYCXTD-C-202004); the Special project of ‘Lingnan modernisation of traditional Chinese medicine’ in 2019 Guangdong Provincial R&D Programme (No: 2020B1111100008); Key Laboratory Programme of Universities in Guangdong Province (No: 2018KSYS006); the College Students’ Innovative Entrepreneurial Training Plan Programme (No: 202210572240) and the Zhaoyang Talent Special Project from Guangdong Provincial Hospital of Chinese Medicine (ZY2022YL15).

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.