Working definition of MCS

Given the different terminology in this field, we established a working definition of MCS for this review:

MCS describes reported adverse reactions to exposure to common chemical agents (solvents, odourants, air pollutants, material, or substances) in doses tolerated by most people. Symptoms involve more than one organ system. Responses are triggered by multiple chemically-unrelated substances.

The criteria are not a case definition for clinical practice and are not meant to be applied to people to determine a diagnosis of MCS. Instead, our approach is to clarify which conditions we will include in the systematic review and which ones we will exclude to ensure transparency.

Search strategy

Literature searches will be designed, executed and documented by an experienced evidence review centre librarian, building on experiences with a scoping review on MCS. The strategy will be peer-reviewed and we will elicit input regarding search terms as well as literature sources. We will search the international literature but restrict to English-language publications, given the differences in taxonomy and nomenclature, as well as differences in cultural norms, that would introduce additional heterogeneity between studies. The basic draft search strategy is shown in the online supplemental appendix. The search will be updated before finalising the review results to ensure that the review is up to date and the search strategy was informed by any additional relevant search terms that were identified in the course of the literature review.

This systematic review addresses multiple aspects of MCS, and the search strategy will cover different research databases to ensure that all scientific literature relevant to MCS has been identified: PubMed (biomedical); CINAHL (nursing and allied health); Embase (biomedical); Web of Science (general scientific database); Scopus (health sciences); PsycINFO (behavioural and social sciences); Healthcare Administration Database (public health administration); Trip Medical Database (health research evidence), Current Contents Connect (multidisciplinary); BIOSIS Citation Index (life sciences); Environment Index (environmental research); Environmental Science Database; Health & Environmental Research Online; SciFinder (chemical literature) and Agricultural & Environmental Science Collection (includes AGRICOLA, environmental research).

In addition, we will identify existing reviews through the systematic review filter in PubMed, the Cochrane Database of Systematic Reviews, the review collection of the Campbell Collaboration and the registry PROSPERO to use these for reference-mining. We will search repositories of practice guidelines including G-I-N,45 MAGICapp,46 Guideline Resources Online47 and the ECRI-maintained guideline database.48 In addition, we will search the US trial registry ClinicalTrials.gov49 and the International Clinical Trials Registry Platform maintained by WHO.50 Finally, we will reference-mine included studies and consult with the TEP to ensure that all relevant literature has been captured.

Eligibility criteria

The eligibility criteria are documented in table 1.

Multiple publications on the same study sample (studies will be defined by the included participants) will be consolidated into one study record to ensure that one study is not counted multiple times regardless of the number of publications reported on the study.

Inclusion screening process

We will use the online database DistillerSR (Evidence Partners, Ottawa, Canada), developed to support systematic reviews to screen the literature search output. The team will design detailed citation and full-text screening forms to ensure a transparent, consistent and unambiguous approach.

Citation screening will be supported by machine learning to ensure that relevant studies are not missed. Citation screening relies on incomplete information (ie, title, abstract, keywords, full citation detail). Therefore, citations perceived to be potentially relevant by at least one of two independent literature reviewers will be obtained as full text for further screening. Full-text screening will apply the detailed eligibility criteria. Training will ensure a shared understanding of all inclusion and exclusion criteria. Full-text publications will be screened by two independent reviewers to reduce errors and bias and any discrepancy will be resolved through discussion in the review team. The screening decisions and reasons for exclusion of studies will be tracked in the online database and citation management software. Citations will be shared with the TEP and will be documented with the review to ensure that the literature flow is transparent and objective. The reasons for exclusion will correspond to the eligibility criteria domains to ensure transparency for the reader.

The literature flow will be documented in a flow diagram, providing information on the number of included and excluded studies at citation and full-text screening and the relative frequency of reasons for exclusion. Studies excluded at full text will be documented in the online supplemental appendix of the review synthesis together with reasons for exclusion. The information will be made publicly available through a data repository to ensure transparency.

Data abstraction

We will design a detailed data extraction form that supports the evidence tables and summary of findings table. Forms for standardised abstractions will be developed in online systematic review software. They will include detailed instructions to reduce ambiguity and coding errors, and forms will be pilot tested. The data abstraction will balance details needed to understand study findings with the need to provide a concise overview of the evidence.

For KQ A (definition), we will abstract the author and date of the publication as well as the name of the professional group endorsing or publishing the definition. We will abstract the terminology used in the publication (eg, MCS, IEI), case definition and diagnostic criteria, as available. If the publication provides any statements regarding how the proposed definition varies from a published definition, we will abstract the information together with any presented rationale for the new definition. We will abstract information on the methods and process of establishing the definition. In addition, we will map the definitions to the Cullen criteria to document similarities and differences (see draft table in the online supplemental appendix). We will document evidence of content, construct and/or criterion validity for the definition. The draft evidence table in the online supplemental appendix provides an outline.

For KQ B (diagnosis), we will abstract the author and publication year, country and the terminology and definition or criteria of MCS. We will document the employed sample, including which characteristics or condition MCS was differentiated from. We will describe the type of test, number of items for scales, details of the test modality and content of the test. The employed reference standard will be described in detail. We will abstract the reliability methods (internal consistency, temporal stability, rater agreement) and results, and any evidence of validity indicating that the test measures what it is supposed to measure (eg, face, construct, content, criterion validity; multitrait-multimethod approaches). Finally, we will abstract the observed diagnostic performance where reported. The table in the online supplemental appendix presents the draft evidence table for the key question.

For KQ C (prevalence and incidence), we will document the year of the estimate, country and sample size. We will abstract the recruiting strategy and data source. We will document the applied case definition of MCS. We will abstract any prevalence and incidence data together with the denominator. Data will be recorded in sufficient detail to allow judgements about the reliability and validity of the estimate. The draft evidence table is presented in the online supplemental appendix.

For KQ D (MCS as a distinct disorder), we will abstract the studies’ aims and general approach undertaken by the authors to address whether MCS should be considered a distinct disorder. We will document the results and the authors’ conclusion and specifically any statement regarding the authors’ conceptual agreement with MSC as a distinct disorder with a differential clinical profile. The draft evidence table in the online supplemental appendix outlines the suggested display of the data.

KQ E (underlying mechanism) will document the study details, suggested biological mechanism, any methods used to test the hypothesis, the type, results, and the strength of evidence.

We will organise the evaluated mechanisms of actions based on published taxonomies.5 41 51 Depending on the identified research, we will differentiate immune system dysregulation, neural sensitisation and hyperresponsivity, chemo-sensitive transient receptor potential function, neurogenic inflammation, limbic system dysfunction, oxidative stress hypothesis, and genetic theories. We will review our categorisation system for the identified mechanisms with the TEP to ensure the clear organisation of the available information.

We will broadly differentiate evidence as direct evidence using MCS samples, mechanistic evidence from animal or lab research, and parallel evidence from samples broader than or related to MCS (eg, samples including participants with different SAEF).52 The data abstraction will provide details for each study describing the empirical evidence and documenting information relevant to weight of evidence criteria such as strength of association, consistency, specificity, temporality, biological gradient, plausibility, coherence, experimental, and analogue evidence.52–54 The draft evidence table is presented in the online supplemental appendix.

We will standardise the strength of association data where possible to facilitate the comparison across studies (eg, calculating measure-independent effect sizes, converting absolute numbers to proportions). We will abstract relevant data for each individual study, for example, documenting consistency if studies reported on multiple participant samples or animal models or different tests. Data related to specificity will include empirical evidence of discriminant validity. We will abstract details of suggested alternative causes or descriptions of MCS symptoms (eg, somatisation disorder) and document which proportion of the MCS symptoms these explain, per study authors. Temporality will report the tested timeframe and the presence and absence of evidence of immediate effects following exposure. Any identified dose-response relationships will be documented in the biological gradient data field. The plausibility field will extract all information reported by the authors indicating a possible biological mechanism underlying the association (eg, chemo-sensitive receptor sensitisation). Coherence will be documented across scientific tests where studies used multiple tests. The evidence table will distinguish experimental and analogue evidence.

KQ F (interventions) will document evaluations of interventions, for example, those that aim to alter the course of the condition or that propose management strategies and treatment. We will abstract the country and study design. We will document the sample, definition or measure of MCS used, the number of participants with MCS and the clinical setting. We will categorise the type of intervention (eg, desensitisation), and provide detail on the intervention content and components, duration and intensity and intervention personnel. In addition, we will describe the comparator type that was used to evaluate the effect of the intervention. We will facilitate the comparison across studies by calculating measure-independent effect sizes and converting absolute numbers to proportions. We will abstract outcome measures used to assess the effect of the intervention in terms of effectiveness and safety, and the reported results for the intervention and the comparator group. The draft evidence table in the online supplemental appendix shows how the results will be documented.

In addition to the displayed information, we will abstract the number of studies in the intervention and control or comparator group, the central tendency (eg, mean) and a measure of dispersion (eg, SD) or counts (eg, proportion of people) for the intervention and the comparator group to prepare data for meta-analysis (see Synthesis section).

Where publications are ambiguous or unclear, we may contact individual authors for clarification. We will export data into summary tables and figures or data files for further analysis.

Critical appraisal

The critical appraisal of individual studies will focus on domains relevant to the study type and review question. We will summarise the results across studies for each KQ, but a detailed table will show the results for each study and criterion (see online supplemental appendix for draft tables).

For KQ A (definition), we will assess the source (eg, evidence of endorsement by a professional organisation). We will assess the processes used to develop the definition by assessing the domain stakeholder involvement (eg, workgroup to establish a definition, panel composition) and whether the definition is evidence-based (eg, based on a systematic review to identify domains, developed in an empirical study comparing prevalence of different definitions). Finally, we will evaluate the validity testing and use (eg, is the definition used in practice by other authors, has it been empirically used since the development). These domains have been applied previously to conceptual work publications.55–60

For the tools relevant to KQ B (diagnosis), we will evaluate four domains: patient selection, index test characteristics, reference standard quality as well as flow and timing.61 For psychometric measurement instruments, we will adapt the Consensus-based Standards for the selection of health Measurement Instruments: reliability, validity, responsiveness, interpretability criteria.62

For KQ C (prevalence and incidence), we will adapt Joanna Briggs Institute (JBI) criteria for critical appraisal of prevalence studies.63 64 The criteria address multiple aspects of study conduct (Was the sample frame appropriate to address the target population? Were study participants sampled in an appropriate way? Was the sample size adequate? Were the study subjects and the setting described in detail? Was the data analysis conducted with sufficient coverage of the identified sample? Were valid methods used for the identification of the condition? Was the condition measured in a standard, reliable way for all participants? Was there appropriate statistical analysis? Was the response rate adequate, and if not, was the low response rate managed appropriately?).

For KQ D (MCS as a distinct disorder), we will assess dimensions of applicability, including regarding participants, intervention/exposure, comparison, measures, setting and study design features.65

For KQ E (underlying mechanism), we will adapt the JBI critical appraisal checklist for analytical cross-sectional studies66 in addition to study design-specific domains. The mechanism of action has been addressed in diverse studies. Nonetheless, it is critical that studies are assessed using common sources of bias such as selection bias and detection bias. We will adapt the following questions: Were the criteria for inclusion in the sample clearly defined? Were the study subjects and the setting described in detail? Was the exposure measured in a valid and reliable way? Were objective, standard criteria used for measurement of the condition? Were confounding factors identified? Were strategies to deal with confounding factors stated? Were the outcomes measured in a valid and reliable way? Was appropriate statistical analysis used?

For KQ F (interventions), critical appraisal will focus on risk of bias, that is, potential distortions of the reported results. We will assess common sources of bias applicable across study designs: selection bias, performance bias, detection bias, attrition bias and reporting bias; compatible with the latest revision of the Cochrane risk of bias tool (RoB2 assessing the randomisation process, deviation from intended interventions, missing outcome data, measurement of outcomes and selection of the reported results).66 We will summarise the results across studies in an overview figure and document results for each study and criterion (see online supplemental appendix for draft).

Synthesis

The synthesis method will be tailored to each key question to ensure that the question can be answered. For all key questions, we will support the comparison and summary across studies by transposing study-specific reported results into units that allow comparison across studies. For this, we will convert raw numbers and absolute values reported in individual studies to rates and proportions (using the numerator and denominator) to make results across studies easier to compare. Furthermore, we will translate effects into measure-independent units (eg, relative risks, standardised mean differences) to facilitate comparisons across studies. In particular for KQ E (underlying mechanism), in addition to the study-level data abstraction, information will be collated for the data integration across studies and type of evidence outlined in the strength of association section of the data abstraction section.

All studies meeting inclusion criteria will be documented in evidence tables. Tables will allow a comprehensive overview of the included studies for each key question, documenting all variables abstracted during data extraction. The evidence tables will be organised by study and will provide sufficient but not overwhelming detail to achieve a concise overview. The results for each key question will be summarised in a narrative synthesis, supported by illustrative figures and tables.

As outlined, for KQ A (definition), structured tables will organise the existing definitions. This will include a component table, which uses the most common published criteria as a reference standard to compare and contrast definitions (see draft in the online supplemental appendix).

KQ B (diagnosis) will document the performance of diagnostic tools. In the absence of sufficient data to calculate diagnostic performance, we will abstract common measures such as sensitivity and specificity as reported by the authors. Given that the measures are not independent from each other, we will plot the sensitivity and specificity in one figure.

Prevalence and incidence data for KQ C (prevalence and incidence) will be documented in figures showing the estimate and CIs based on the SE for each identified study. The figures will provide an overview of the estimate for each study and the size of the study, the range of estimates across studies, the central tendency across studies and any outliers. The figure will be grouped by country. The US studies are a prespecified subgroup. Depending on the number of identified studies, a summary figure or table will order the estimate by publication year. Where estimates differ across studies for the same setting, we will explore potential sources of heterogeneity and document these together with the estimate.

KQ D (MCS as a distinct disorder) will document the results of research assessing MCS as a distinct disorder in a comprehensive table. In addition, we will provide a summary table that documents the aim and approach together with the authors’ conclusion of the study. To the extent possible, the table will highlight whether the authors concluded that MCS is a distinct disorder or not. The online supplemental appendix shows a draft summary table. The summary of findings table will summarise the approaches and results of studies that concluded MCS is a distinct disorder, those that concluded that MCS is not a distinct disorder and those that did not come to a definitive conclusion.

The summary of findings table for KQ E will be organised by the evaluated mechanism potentially underlying MCS and collate the existing research evidence for each mechanism. The synthesis may need to be further stratified by MCS symptom.

For KQ F (interventions), effect sizes will be statistically pooled to provide a numerical estimate of the size of the treatment effect where possible. Meta-analysis is a key data aggregation method, which will allow small studies to meaningfully contribute to the evidence base even when the individual study was originally too small to show a statistically significant effect on its own. Meta-analysis will also be sensitive to small effects of interventions (small effects can occur when not everyone in a sample is affected or everyone experiences only a small change). Analyses will be conducted in the statistical program R, applying current and appropriate methods such as Hartung-Knapp corrections when only a small number of studies overall or within intervention categories is available.67 Individual and pooled results will be documented in forest plots allowing a clear overview of the effects, study size and uncertainty surrounding the treatment effect, as well as the general tendency, outliers and variation across studies. Heterogeneity will be documented with the I² statistic. We will explore sources of heterogeneity in meta-regressions and subgroup analyses. A prespecified subgroup is whether the study addresses management (eg, coping with MCS) or treatment (eg, addressing the underlying mechanism). The code and the results will be generated in an R Markdown file to allow complete transparency of the synthesis methods. We will start with an overall effect before drilling deeper (first lumping, then splitting). Results may be presented in multiple forest plots or in stratified forest plots by broad intervention categories to organise the research studies depending on the identified research volume.

We will aim to come to a summary rating for studies that captures multiple outcomes to facilitate the use of the critical appraisal results. The assessment will inform the interpretation of the results of the identified studies. The ratings and other study limitations will be incorporated into the quality of evidence assessment across studies. The rating will be drafted by one reviewer and discussed in the review team to ensure transparency and consistency. The summary of findings table will be organised by key question and outcome. It will show the study design and number of studies, any reasons for downgrading the quality of evidence, the findings across publications and the overall Grading of Recommendations, Assessment, Development and Evaluations (GRADE) category. The draft summary of findings table in the online supplemental appendix provides a broad outline.

The review team will synthesise evidence across studies, assess the certainty of evidence and clearly communicate the team’s confidence in the summary results. We will first establish a hierarchy of outcomes for each topic area to ensure that evidence is summarised objectively and comprehensively. In line with GRADE methodology, we may limit outcomes per key question to ensure clear and transparent results. The outcomes will be informed by the TEP, published reviews, identified studies and clinical expertise. The outcomes will be selected before the results of the studies are known to ensure an unbiased analysis. Any statistically significant effects will be translated from relative effects to absolute effects, and standardised mean differences into mean differences on a known scale, to support the interpretation of the results.

The certainty of evidence assessment across studies will use transparent criteria to downgrade (study limitations, inconsistency, indirectness, imprecision, publication bias) or upgrade (large effect, dose-response relationship, confounding reduces effect) the quality of evidence. We will use published certainty of evidence criteria and select the most appropriate guidance for the available data relevant to each key question.68 69 For example, the GRADE adaptation for prognostic studies also considers the phase of investigation.70 For KQ A to E, we will not use randomised controlled trials (RCTs) as the starting point from which the quality of evidence is either upgraded or downgraded, given that an RCT is an unlikely study design to address these key questions. The study design of identified research initially determines a hierarchical grade of the evidence. In studies assessing clinical interventions, RCTs are initially considered high-quality evidence, whereas other study designs are initially classified as low-quality evidence. Applying standard clinical research evidence hierarchies to research files where key study designs are missing or considered inappropriate reduces the ability to differentiate the certainty of evidence. It creates an artificial floor effect because initially high quality of evidence does not exist and evidence is likely to be systematically rated low or very low.71 Standard criteria will be applied to KQ F (interventions) to adhere to intervention research standards. For each evidence statement, we will communicate our confidence in the summary estimates using internationally recognised categories:

• High indicates that we are very confident that an effect estimate lies close to the true effect for a given outcome, as the body of evidence has few or no deficiencies. As such, we believe the findings are stable: that is, further research is very unlikely to change confidence in the effect estimate.

• Moderate indicates that we are moderately confident that an effect estimate lies close to the true effect for a given outcome, as the body of evidence has some deficiencies. As such, we believe that the findings are likely to be stable, but further research may change confidence in the effect estimate and may even change the estimate.

• Low indicates that we have limited confidence that an effect estimate lies close to the true effect for a given outcome, as the body of evidence has major or numerous (or both) deficiencies. As such, we believe that additional evidence is needed before concluding either that the findings are stable or that the effect estimate lies close to the true effect.

• Very low indicates that we have very little confidence that an effect estimate lies close to the true effect for a given outcome, as the body of evidence has very major deficiencies. As such, the true effect is likely to be substantially different from the estimated effect; thus, any estimate of effect is very uncertain.

Throughout the review, results will be interpreted with caution and in the light of the existing research. For comparative effectiveness, statements that do not show a statistically significant effect, we will take evidence of statistical power to detect differences into account before making non-inferiority statements for interventions. Similarly, the interpretation of adverse events will take into account that frequentist approaches are problematic for rare adverse events (rare events require large samples to detect effects). The findings will be summarised in structured narratives, highlighting research results that have been replicated in more than one study.