You are here

  • Home
  • Archive
  • Volume 9, Issue 10
  • An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature

Article Text

Article menu

This article has a correction. Please see:

Download PDFPDF

XML
Ethics
Original research
An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature
  1. Alison Avenell1,
  2. Fiona Stewart1,
  3. Andrew Grey2,
  4. Greg Gamble2,
  5. Mark Bolland2
  1. 1 Health Services Research Unit, University of Aberdeen, Aberdeen, UK
  2. 2 Department of Medicine, University of Auckland, Auckland, New Zealand
  1. Correspondence to Prof Alison Avenell; a.avenell{at}abdn.ac.uk

Abstract

Objective Analyses of the impact of a body of clinical trial reports subject to research misconduct have been few. Our objective was to examine the impact on clinically relevant research of a group of researchers’ trial reports (‘affected trial reports’) affected by research misconduct, and whether identification of misconduct invoked a reappraisal.

Design In 2016, we used five databases and search engines to identify ‘citing publications’, that is, guidelines, systematic and other reviews, and clinical trials citing any of 12 affected trial reports, published 1998–2011, eventually retracted for research misconduct. The affected trial reports were assessed more likely to have had impact because they had hip fracture outcomes and were in journals with impact factor >4. Two authors assessed whether findings of the citing publications would change if the affected trial reports were removed. In 2018, we searched for evidence that the citing publications had undertaken a reassessment as a result of the potential influence of the affected trial reports.

Results By 2016 the affected trial reports were cited in 1158 publications, including 68 systematic reviews, meta-analyses, narrative reviews, guidelines and clinical trials. We judged that 13 guidelines, systematic or other reviews would likely change their findings if the affected trial reports were removed, and in another eight it was unclear if findings would change. By 2018, only one of the 68 citing publications, a systematic review, appeared to have undertaken a reassessment, which led to a correction.

Conclusions We found evidence that this group of affected trial reports distorted the evidence base. Correction of these distortions is slow, uncoordinated and inconsistent. Unless there is a rapid, systematic, coordinated approach by bibliographic databases, authors, journals and publishers to mitigate the impact of known cases of research misconduct, patients, other researchers and their funders may continue to be adversely affected.

  • scientific misconduct
  • impact
  • randomised controlled trials
  • hip fractures

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi-org.ezproxy.u-pec.fr/10.1136/bmjopen-2019-031909

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • A wide-ranging literature search examined the impact of 12 randomised clinical trial reports affected by misconduct.

    • A detailed examination of the extent and effect of these trial reports on guidelines, systematic and other reviews, and clinical trials was undertaken.

    • We only examined the impact of 12/27 retracted trial reports, and assessing the impact on citing publications would have been strengthened if we had contacted the authors of the 68 citing publications.

    • We only examined the effect on published research we were able to identify, and probably have not found all publications, especially guidelines. We did not examine impact on other forms of influence, for example, grant applications, drug company documents.

    Background

    We raised concerns about 33 randomised controlled trial (RCT) reports, ‘affected trial reports’, from one research group in Japan (see online supplementary appendix for list of 33 RCTs).1 2 Our systematic review published in November 2016 examined these affected trial reports published in the field of osteoporosis over 15 years. The affected trial reports ostensibly involved large numbers of older patients with significant comorbidities, such as stroke, Alzheimer’s disease and Parkinson’s disease.1 In September 2016, the editor of the journal that published our systematic review conveyed the results of its investigations to all the journals with affected trial reports. By May 2019, 27/33 of these affected trial reports had been retracted for reasons including fabrication, plagiarism, authorship misconduct and unresolved concerns about data integrity.

    Supplemental material

    Retraction of a research paper may have important implications for clinical practice and present and future research initiatives. Patients and research participants may be put at risk if decisions are based on findings that are later retracted because they were incorrect or unreliable.3 4 It is therefore important to determine the extent of a retracted paper’s influence, for example, through citations in other influential publications, such as systematic reviews and guidelines, and its use in initiating new research. There is evidence that authors of publications that cite retracted work remain unaware of the retraction,5 and this has potentially important consequences for their work, that of subsequent researchers, and for clinical practitioners and patients.

    Analyses of the impact of a body of clinical trial reports subject to research misconduct have been few. Our objective was to examine the impact and influence of a selection of the published affected trial reports most likely to affect clinical guidance and practice and further research. We focused on affected trial reports with hip fracture outcome data in influential journals.

    Methods

    Search criteria

    We studied the impact of a subgroup of the 33 affected trial reports whose integrity was analysed in our systematic review.1 This subgroup of trial reports was used because these trials had hip fracture as an outcome, arguably the most important consequence of osteoporosis, and affected trial reports on this outcome are likely to have the greatest impact. We included all affected trial reports with hip fracture outcomes that had also been published in higher impact journals (ISI Web of Knowledge impact factor >4).

    Evidence identification

    In August 2016, we used Scopus and Web of Science to find citations of each affected trial report and the type of publication that cited each report (‘citing publications’—guidelines, systematic and other reviews, and clinical trials). We also searched Google Scholar, PubMed and personal databases to identify systematic reviews, meta-analyses, narrative reviews and guidelines relating to hip fracture prevention, which potentially would include these affected trial reports. Finally, we sought other types of publications that cited the affected trial reports, through an iterative process, for example, using the following search command in Ovid MEDLINE: (sato.tw) and ((letter or comment$).pt). We excluded self-citing publications by authors of affected trial reports from our evaluations.

    Assessment of impact

    Where possible, meta-analyses which included data from affected trial reports were reanalysed to investigate whether the quantitative findings, such as summary risk ratios in forest plots, would change without the inclusion of those data. In the case of reviews in which data from affected trial reports were not included in quantitative synthesis, we used our judgement. One investigator (FS) initially assessed all citing publications for the influence of affected trial reports, which were then discussed in depth with a second investigator (AA). Agreement was reached between AA and FS on all affected publications, apart from two where AG and MB provided input leading to consensus. We categorised affected publications according to the likelihood of a change in findings if the affected trial reports were excluded:

    1. Findings likely to change.

    2. Uncertain if findings would change.

    3. Findings unlikely to change.

    In November 2018, we searched again Web of Science, Scopus or guideline websites to see if the affected systematic reviews, meta-analyses, narrative reviews and guidelines, identified in August 2016, had published any notice, update, correction or retraction on publishers’ websites resulting from recognition that the publication was potentially influenced by the affected trial reports. We searched Web of Science, or Scopus if not included in Web of Science, to identify the number of times the citing publications we had judged likely or possibly to have been influenced by affected trial reports had themselves ever been cited, and the date of the most recent citation.

    In July 2019, we searched Web of Science for any publication that cited the affected trial reports after they had been retracted, to examine whether these publications mentioned that the affected trial reports had been retracted.

    Patient and public involvement

    We did not involve patients or the public in our work.

    Results

    Twelve trial reports from the original 33 were identified by us for evaluation. These 12 affected trial reports all had hip fracture outcomes and were published between 1997 and 2011 in journals with impact factors>4, with 3182 reported participants (table 1).6–17 They were published in journals with a median impact factor of 5.8 (range from 4.57 9 10 to 3014). All 12 affected trial reports were retracted between June 2016 and April 2019, but by July 2019 only 7 (58%) were marked as retractions on both Ovid Medline and PubMed,9 10 12–15 17 and two further affected trial reports were marked as retracted on PubMed but not on Medline.11 16

    View this table:
    Table 1

    Affected trial reports in journals with impact factor >4 and hip fracture outcome data

    We examined 40 publications in July 2019 that cited any of the 12 affected trial reports after they were retracted. Thirty-four publications (85%) expressed no concern about the affected trial reports, and six (15%) cited the affected trial reports but discounted their findings as a result of misconduct.

    Citations of affected trial reports

    By August 2016, the 12 affected trial reports were cited a total of 1158 times in publications of any kind, identified by our literature searches. The median number of citations for affected trial reports was 84 (range 14 to 323).

    Sixty-eight systematic reviews, meta-analyses, narrative reviews, guidelines and clinical trials cited at least one of the 12 affected trial reports. Each affected trial report was cited by a median of 11 of the 68 publications (range one to 25). Five citing publications, including Agency for Healthcare Research and Quality (AHRQ) comparative effectiveness reviews, were not listed on Web of Science or Scopus. Of the 68 citing publications indexed on Ovid Medline, 27 were systematic reviews, meta-analyses and narrative reviews, 9 effectiveness reviews and guidelines, and 32 clinical trial reports.

    Reviews and meta-analyses

    The 12 affected trial reports were included in 23 systematic reviews, meta-analyses and narrative reviews, covering a broad spectrum of topics, including prevention of falls and fractures, treatment of psychiatric symptoms and the role of homocysteine in disease.18–43 Four further reviews and meta-analyses cited but did not include any data from affected trial reports in their analyses.44–47

    Fracture reviews and meta-analyses

    Nine reviews and meta-analyses relating to hip fracture prevention were identified that cited at least one affected trial report. The findings of four were likely to change following the removal of the affected trial reports (see table 2).18 21 22 24

    View this table:
    Table 2

    Number of reports citing affected trial reports, with assessment of impact of trial reports

    Two systematic review authors did not express caution that their findings were derived from one group of investigators. The systematic review by Zhang et al 21 (three citations, most recent December 2016) only included affected trial reports.21 However, the authors noted the lack of generalisability from Japanese-only populations. The systematic review by Zhao et al 22 focused on hip fracture and bone mineral density outcomes in Alzheimer’s disease; affected trial reports were the only sources of bone mineral density data.

    Cockayne et al 18 undertook a meta-analysis of vitamin K for fracture prevention (217 citations, August 2018) which influenced Japanese osteoporosis guidelines. The reduction in hip fractures was statistically and clinically significant with an OR of 0.23 and narrow CI (95% CI 0.12 to 0.47). However, Cockayne et al 18 also included a sensitivity analysis to investigate the effect of removing the three affected trial reports.8–10 This analysis changed the result to a statistically non-significant result with wide confidence intervals (OR 0.30, 95% CI 0.05 to 1.74). The reason given for conducting the sensitivity analysis was that the trial populations were from a single centre and included participants at much higher risk of fractures than other trials. The authors expressed some caution when interpreting the main findings of their review because of the uncertainty introduced by this sensitivity analysis and their conclusions—that vitamin K helps to prevent hip fractures—would be different if the affected trial reports were omitted. Importantly, this 2006 meta-analysis, without any caveat related to the sensitivity analysis, is the sole evidence cited for vitamin K preventing vertebral and non-vertebral fractures in the journal publication of the 2011 Japanese guidelines for osteoporosis23 (122 citations, October 2018). In 2018, in response to retractions, Cockayne’s group published a letter of explanation and corrected article,19 20 removing the three affected trial reports, with the revised OR for hip fracture of 0.30 (95% CI 0.05 to 1.74).

    One affected trial report14 was judged to have influenced the strength of a review’s conclusions. This was a narrative review of B vitamins and bone health24 (eight citations, September 2018). The affected trial report14 showed that B vitamins significantly reduced hip fractures, contrary to the evidence cited that most studies did not demonstrate reduced fracture risk. The authors noted that the results of one affected trial report14 were unusual and speculated that improvements in neurological and cognitive function from B vitamins would prevent fall-related fractures. We judged that without the affected trial report the review’s conclusions of lack of efficacy of the intervention would be stronger.

    Cases where we were unable to reanalyse meta-analyses after removal of affected trial reports would have been facilitated by authors providing open access to all their data. For four meta-analyses, it was unclear if omission of the affected trial reports would alter the findings (Richy et al 25 79 citations, October 2018; Richy et al 26 91 citations, February 2018; Murad et al 27 26 citations, August 2018; Yang et al 28 54 citations, October 2018). Clarification of the impact of the affected trial reports requires the reviews’ authors to repeat their meta-analyses with and without the affected trial reports. The citation of one affected trial report11 in the review by McCarus et al 29 is little more than a passing reference and data from the trial report were not used.

    Falls reviews and meta-analyses

    Two affected reviews and meta-analyses related to the prevention of falls were identified, since the affected trial reports also provided data on falls.

    The results from one affected trial report10 changed the findings for a combined treatment (calcium, vitamin D and vitamin K for falls prevention). One Cochrane review on the prevention of falls in the community30 (756 citations, November 2018) included an unpooled meta-analysis of data from one affected trial report10 and one other trial of calcium alone, relating to the number of fractures caused by falling. The analysis shows a large, statistically significant, reduction in fracture risk in the intervention group from the affected trial report (risk ratio 0.13, 95% CI 0.04 to 0.43), and a null effect in the other trial48 (risk ratio 0.90, 95% CI 0.69 to 1.16).

    Data from two affected trial reports were included in unpooled meta-analyses in the review by Batchelor et al 31 (65 citations, September 2018), in which the affected trial report data were not outlying.

    Other reviews and meta-analyses

    Twelve other affected reviews and meta-analyses were identified. Removing affected trial reports from three would likely alter their conclusions. The conclusion of one systematic review on interventions for osteoporosis (Hermann et al,32 65 citations, 2018) that B-vitamins were likely to reduce the risk of osteoporosis was supported by data from an affected trial report,14 which showed a reduction in hip fractures in the intervention group. The review’s authors note several limitations in the affected trial report, but commented on its ‘very promising’ results.

    In their review of vitamin D and Parkinson’s disease, Peterson et al 33 (16 citations, 2017) base their conclusions almost entirely on data from four affected trial reports.8 9 15 16

    Three affected trial reports8 15 16 were cited in the review by Binks and Dobson34 (one citation, 2017) as evidence for the benefit of vitamin D and bisphosphonates in people with Parkinson’s disease. Although Binks and Dobson were careful to draw attention to the limitations of the trial reports, nonetheless their conclusions would be substantially different without these data.

    Affected trial reports were included in three reviews (Alibhai et al 35 118 citations, October 2018; Carda et al 36 24 citations, September 2018; Simpson et al 37 14 citations, April 2018) where it was unclear if findings would be altered by the omission of the affected reports’ data. The conclusions of six systematic reviews were unlikely to change if data from affected trial reports were omitted.38–43

    Systematic reviews excluding affected trial reports

    A further four systematic reviews cited but did not include affected trial reports in their reviews as a result of existing concerns with data,44–46 or awaiting responses to enquiries about data.47 One was a Cochrane review by one of the authors of this paper, with concerns dating back to 2006.44 Another Cochrane review, whose authors corresponded with AA, excluded trials for not fitting study inclusion criteria.45 Latham et al 46 appeared to exclude one trial8 because of its poor quality from their review of vitamin D for falls prevention and other outcomes. Verheyden et al 47 categorised two affected trial reports as awaiting assessment11 12 in their Cochrane review of falls prevention after stroke.

    Effectiveness reviews and guidelines

    Affected trial reports were cited in nine effectiveness reviews and clinical guidelines (one published in Scotland, the others in the USA), for stroke,49 fracture prevention50–55 and fall and injury prevention.56 Removing these affected trial reports would likely alter findings in five reviews and guidelines.51–54 56

    The effectiveness review from the US AHRQ in 2007 on fracture prevention51 (no citation count available) included six affected trial reports in their Table 56,6 8 11–13 15 which are the only trials cited for bisphosphonates preventing fractures in high risk falls patients. In addition, three affected trial reports11–13 are the only evidence used to support the 2.5 mg dose of risedronate for preventing hip fracture. This dose of risedronate does not have marketing approval in the USA (https://www.fda.gov/downloads/drugs/developmentapprovalprocess/ucm071436.pdf), but does in Japan (https://www.ajinomoto.com/en/presscenter/press/detail/g2009_07_31.html).

    The publication in the Annals of Internal Medicine 52 from this AHRQ review has been cited 346 times, including September 2018, and it references four of the six above-mentioned affected trial reports, with these reports being the sole sources of data evidencing the reduction in fractures from bisphosphonates in patients with Parkinson’s disease, Alzheimer’s disease or stroke. The linked guideline from the American College of Physicians53 (114 citations, March 2018) references the same four affected trial reports as evidence for bisphosphonate use in populations at increased risk of falls.

    When the AHRQ review was updated in 201254 (no citation count available), it included evidence from five affected trial reports,11–13 15 16 with no new trials from other authors providing data for risedronate 2.5 mg/day in the prevention of hip fracture. The effectiveness review also states that this dose is equivalent to higher doses of risedronate.

    A 2008 evidence-based handbook for nurses56 (no citation count available) contains the statement that risedronate is effective in preventing fractures in older women, older men who have had a stroke and older women with Alzheimer’s disease, based entirely on two affected trial reports.12 13

    It was unclear whether exclusion of the affected trial reports would alter findings in one report. American stroke guidelines49 (1230 citations, November 2018) used evidence from one report14 of vitamin B12 and folate supplementation as the only evidence when discussing fracture prevention among patients with a recent ischaemic stroke. However, ‘routine’ supplementation of vitamins was not recommended, so we judged that it was unclear if findings, related to higher risk patients, would change without this one report.

    Findings of three reviews were unlikely to change following exclusion of affected trial reports. The updated 2017 American College of Physicians’ guidelines55 (74 citations, October 2018) includes two of the affected trial reports on 2.5 mg daily risedronate13 16 in its overview of the evidence for the use of risedronate from the AHRQ review,54 but does not discuss the specific issue of the lower dose of risedronate. Guidelines from Scotland relating to osteoporosis and fractures50 express caution about using the affected trial report on vitamin B12 and folate supplementation14 in recommendations: ‘As this was a Japanese population that had suffered a stroke, it is not certain how relevant the findings are to a Scottish population.’ A guideline from AHRQ57 excluded one trial report8 from its review of interventions to prevent falls in older people. The reason for exclusion was that the report did not focus on the outcome of interest, that is, the rate of falls or number of fallers, despite what appeared to be relevant falls data in the affected trial report.

    Trials

    We identified 32 clinical trial reports (including 27 RCTs) which cited affected trial reports. In eight cases,58–65 affected trial reports contributed to the rationale for undertaking further RCTs. These RCTs are listed in table 3. Seven trials discussed one or more of the affected trial reports in their introduction sections, and five trials in their discussion sections. The strongest suggestion of influence in study design or rationale comes from the RCT by van Wijngaarden et al,65 published in 2014, which discusses two RCTs in people at risk of cardiovascular disease or with cerebrovascular disease which had been unable to demonstrate B vitamins preventing fractures. These RCTs were contrasted with the affected trial report,14 which reported a reduction in hip fractures in stroke survivors. van Wijngaarden et al then state that ‘Given the conflicting results and low generalizability to the general older population, further investigation is needed.’ van Wijngaarden et al’s trial randomised 2919 participants to B vitamins or placebo for 2 years, and found no treatment effect on osteoporotic fractures.65

    View this table:
    Table 3

    RCT reports in which affected publications by Sato and colleagues are included in the justification for the trial

    In another eight RCTs (not shown in table 3), the authors cited affected trial reports to draw attention to the disparities between their own findings and those reported.66–73 It appeared unlikely that the affected trial reports contributed to the rationale for these trials.

    Discussion

    Our analysis suggests that affected trial reports are likely to have had an adverse impact on clinical care and other research. By 2016, affected trial reports were widely cited in the published literature of particular relevance to older people with Parkinson’s disease, stroke or Alzheimer’s disease, where, despite their generally small sample size and number of events, they dominated the literature for fracture prevention. Despite recommendations for caution in deriving conclusions from data from a very limited number of authors and centres,74 75 authors of reviews that included affected trials rarely expressed caution.21 22 We were unable to identify published or registered (ClinicalTrials.gov) RCTs of bisphosphonates in these patient groups by other research groups. Thus, other researchers (and funders) may have been dissuaded from undertaking further trials by evidence from these affected trial reports. It was apparent that some systematic reviews and guidelines, particularly for the earlier three patient groups, would be different for vitamin K and risedronate 2.5 mg/day if the affected trial reports were removed, and that some affected systematic reviews and guidelines have themselves been widely disseminated.18 23 30 49 52 However, we do not know which parts of these systematic reviews and guidelines have been influential. With one exception,18–20 authors and/or journals of citing publications have either not identified that their publications have been compromised, or decided no action is required, although the latter seems unlikely. To our knowledge, bibliographic database/journal/publisher/guideline developer structures are not established that permit systematic identification and correction of publications that are affected by the inclusion of research with compromised integrity. Even if removing the affected trial reports did not influence their conclusions, citing authors should publish an update. This should give details of their examination of the impact of the correction or retraction on their own work, and confirm that changes are not required or have been made. This would remove uncertainty in the interpretation of their work.76 77 This could be aided by publishing an amended article, with an updated version number, as has been suggested by Barbour and colleagues.78

    Our assessment in August 2016 relates to publications up to that time. New, affected publications have continued to accumulate. We only assessed the impact of the 12 likely most influential affected trial reports (based on hip fracture outcomes and publication in journals with impact factor >4) from the 33 we originally investigated.1 The remaining 21 affected trial reports may also have been influential. For example, the 2007 AHRQ report by MacLean and colleagues51 on treatments to prevent fractures includes six affected trial reports that we did not assess. It was not always possible to fully assess the impact of affected trial reports, because published data in affected publications were insufficient to allow us to replicate analyses after excluding affected trial reports. Examining impact in a network meta-analysis such as that by Murad and colleagues27 would be difficult, even if data were available. Narrative reviews can be particularly vulnerable to studies with research misconduct,75 and assessing impact in narrative reviews was often more challenging, as others have found.74

    We only investigated affected trial reports’ impact on published research. They could also have influenced grant applications, educational events, media coverage and social media, evaluation of which require a very broad range of information sources. Most importantly, we could not directly establish the effect on patients from clinical practices informed by the unreliable research. We did not examine the impact of reviews and systematic reviews authored by the group of researchers who published the affected trial reports, which includes more than 30 reviews and meta-analyses. Such active dissemination by self-citation in cases of prolific misconduct also occurred in the Reuben and Fujii cases.74 79

    We have probably missed guidelines in our evaluation of citing publications, since these are poorly covered by indexing databases. AHRQ full guidelines51 54 were identified through linked journal articles, and Scottish Intercollegiate Guidelines Network (SIGN) guidelines from personal databases.50 Thus, we have probably underestimated the impact of these 12 trial reports.

    Our findings are consistent with those of others who have investigated the impact of publications affected by research misconduct on subsequent publications and systematic reviews.74 76 80–83 In the Scott Reuben case, almost half of Reuben’s articles on perioperative analgaesia were still being cited more than 5 years after their retraction,81 and his reports widely infiltrated literature in this area.84

    Retractions of affected trial reports examined here started only in 2016, but concerns about research by this Japanese research group had been expressed as early as 2004–2007 by other groups, so that delays in investigation also increased the impact of this misconduct.85–89 Mott and colleagues found a 46% reduction in citations of randomised clinical trial reports in the first year after retraction,83 and retractions also reduce subsequent publication by authors associated with misconduct.90

    It seems systems have not changed to mitigate the impact of misconduct, once it is identified, more than 10 years since these issues were highlighted by Sox and Rennie.76 van der Vet and Nijveen91 argued on the basis of a single, preclinical case study that indirect citations did not contribute to the propagation of research misconduct. However, for randomised trials in clinical areas affecting systematic reviews and guidelines further propagation is likely, as we show in the case of the systematic review by Cockayne et al 18 and its influence on Japanese osteoporosis guidelines.23 In the case of Fujii’s extensive publications, the effect of his misconduct on the management of postoperative nausea and vomiting appears to have only been minimised by the large volume of publications from other authors.79 In a recent paper, analyses by Fanelli and Moher92 suggested that meta-analyses may overestimate their summary effect sizes when they include studies later retracted for issues with data, methods or results.

    Delays in the processes of investigating, correcting or retracting research misconduct add to the impact on patients, funders and other researchers. Delays in retraction by journals, even in response to official notification by investigating authorities, continue to be problematic and contribute to the impact of retracted work.93 Once a retraction is posted by a journal all bibliographic databases and search engines should be swiftly updated. This was not the case with affected trial reports, which were retracted but not always listed as retracted on Ovid Medline and PubMed, in some cases more than 2 years later. Journals and their publishers could help to prevent the citation of retracted studies by themselves checking or requiring authors to check their reference list against Retraction Watch’s database (http://retractiondatabase.org/RetractionSearch.aspx?) before submission.77 Zotero software that is linked to Retraction Watch’s database,94 or ReTracker linked to Retractions in PubMed,95 might facilitate authors’ awareness of retractions. Clearly marked, retracted articles and properly informative retraction notices should be linked on journals’ websites and both should be freely accessible.76 80

    Research misconduct can have widespread detrimental effects on subsequent research initiatives and clinical practice. Some possible solutions to minimise the impact of retracted publications are given in box 1, but there remains no overarching body with the commitment to coordinate managing the consequences of proven research misconduct.

    Box 1

    Some possible solutions for minimising the impact of retracted research reports

    • Journals and publishers should ensure that expressions of concern, retractions or corrections are appropriately flagged so that they are immediately available to be listed as such on bibliographic databases, including that of Retraction Watch, and search engines.

    • Publishers should sign up to The CrossMark (https://www.crossref.org/services/crossmark/), an initiative to take readers to the current version of the paper, which should include expressions of concern, retractions or corrections.

    • After institutional investigations have found that misconduct has taken place, institutions could notify corresponding, first authors and senior authors of citing publications.

    • Listing an expression of concern, retraction or correction on bibliographic databases should generate automatic alerts to corresponding, first authors and senior authors of citing publications.

    • Retraction Watch’s database of retractions, linked to reference management software, should be used to regularly scan researcher’s personal reference libraries.94

    • Journals and their publishers could help to prevent inappropriate citations by themselves checking or requiring authors to check their reference list for expressions of concern, retractions or corrections.

    • Organisations responsible for publications, which are not usually listed on bibliographic databases, e.g. clinical guideline groups, should regularly check Retraction Watch’s database against their reference lists, or ensure their guidelines are listed on bibliographic databases.

    • Authors of citing publications should publish an amendment, or a reassurance that the publication is unaffected, with a link to the affected publication.

    References

    1. 1.↵
      2. Bolland MJ ,
      3. Avenell A ,
      4. Gamble GD , et al
      . Systematic review and statistical analysis of the integrity of 33 randomized controlled trials. Neurology 2016;87:2391–402.doi:10.1212/WNL.0000000000003387
      OpenUrlCrossRef
    2. 2.↵
      2. Gross RA , Editor-in-Chief
      . Statistics and the detection of scientific misconduct. Neurology 2016;87:2388.doi:10.1212/WNL.0000000000003390
    3. 3.↵
      2. Steen RG
      . Retractions in the medical literature: how can patients be protected from risk? J Med Ethics 2012;38:228–32.doi:10.1136/medethics-2011-100184
      OpenUrlAbstract/FREE Full Text
    4. 4.↵
      2. Marcus A
      . A scientist's fraudulent studies put patients at risk. Science 2018;362:394.doi:10.1126/science.362.6413.394-a
    5. 5.↵
      2. Davis PM
      . The persistence of error: a study of retracted articles on the Internet and in personal libraries. J Med Libr Assoc 2012;100:184–9.doi:10.3163/1536-5050.100.3.008
      OpenUrlPubMed
    6. 6.↵
      2. Sato Y ,
      3. Maruoka H ,
      4. Oizumi K
      . Amelioration of hemiplegia-associated osteopenia more than 4 years after stroke by 1 alpha-hydroxyvitamin D3 and calcium supplementation. Stroke 1997;28:736–9.doi:10.1161/01.STR.28.4.736
      OpenUrlAbstract/FREE Full Text
    7. 7.↵
      2. Sato Y ,
      3. Honda Y ,
      4. Kuno H , et al
      . Menatetrenone ameliorates osteopenia in disuse-affected limbs of vitamin D- and K-deficient stroke patients. Bone 1998;23:291–6.doi:10.1016/S8756-3282(98)00108-2
      OpenUrlCrossRefPubMedWeb of Science
    8. 8.↵
      2. Sato Y ,
      3. Manabe S ,
      4. Kuno H , et al
      . Amelioration of osteopenia and hypovitaminosis D by 1alpha-hydroxyvitamin D3 in elderly patients with Parkinson's disease. J Neurol Neurosurg Psychiatry 1999;66:64–8.doi:10.1136/jnnp.66.1.64
      OpenUrlAbstract/FREE Full Text
    9. 9.↵
      2. Sato Y ,
      3. Honda Y ,
      4. Kaji M , et al
      . Amelioration of osteoporosis by menatetrenone in elderly female Parkinson's disease patients with vitamin D deficiency. Bone 2002;31:114–8.doi:10.1016/S8756-3282(02)00783-4
      OpenUrlCrossRefPubMedWeb of Science
    10. 10.↵
      2. Sato Y ,
      3. Kanoko T ,
      4. Satoh K , et al
      . Menatetrenone and vitamin D2 with calcium supplements prevent nonvertebral fracture in elderly women with Alzheimer's disease. Bone 2005;36:61–8.doi:10.1016/j.bone.2004.09.018
      OpenUrlCrossRefPubMedWeb of Science
    11. 11.↵
      2. Sato Y ,
      3. Iwamoto J ,
      4. Kanoko T , et al
      . Risedronate therapy for prevention of hip fracture after stroke in elderly women. Neurology 2005;64:811–6.doi:10.1212/01.WNL.0000152871.65027.76
      OpenUrlCrossRefPubMed
    12. 12.↵
      2. Sato Y ,
      3. Iwamoto J ,
      4. Kanoko T , et al
      . Risedronate sodium therapy for prevention of hip fracture in men 65 years or older after stroke. Arch Intern Med 2005;165:1743–8.doi:10.1001/archinte.165.15.1743
      OpenUrlCrossRefPubMedWeb of Science
    13. 13.↵
      2. Sato Y ,
      3. Kanoko T ,
      4. Satoh K , et al
      . The prevention of hip fracture with risedronate and ergocalciferol plus calcium supplementation in elderly women with Alzheimer disease: a randomized controlled trial. Arch Intern Med 2005;165:1737–42.doi:10.1001/archinte.165.15.1737
      OpenUrlCrossRefPubMedWeb of Science
    14. 14.↵
      2. Sato Y ,
      3. Honda Y ,
      4. Iwamoto J , et al
      . Effect of folate and mecobalamin on hip fractures in patients with stroke: a randomized controlled trial. JAMA 2005;293:1082–8.doi:10.1001/jama.293.9.1082
      OpenUrlCrossRefPubMedWeb of Science
    15. 15.↵
      2. Sato Y ,
      3. Iwamoto J ,
      4. Kanoko T , et al
      . Alendronate and vitamin D2 for prevention of hip fracture in Parkinson's disease: a randomized controlled trial. Mov Disord 2006;21:924–9.doi:10.1002/mds.20825
      OpenUrlCrossRefPubMedWeb of Science
    16. 16.↵
      2. Sato Y ,
      3. Honda Y ,
      4. Iwamoto J
      . Risedronate and ergocalciferol prevent hip fracture in elderly men with Parkinson disease. Neurology 2007;68:911–5.doi:10.1212/01.wnl.0000257089.50476.92
      OpenUrlCrossRefPubMed
    17. 17.↵
      2. Sato Y ,
      3. Iwamoto J ,
      4. Honda Y
      . Once-weekly risedronate for prevention of hip fracture in women with Parkinson's disease: a randomised controlled trial. J Neurol Neurosurg Psychiatry 2011;82:1390–3.doi:10.1136/jnnp.2011.244574
      OpenUrlAbstract/FREE Full Text
    18. 18.↵
      2. Cockayne S ,
      3. Adamson J ,
      4. Lanham-New S , et al
      . Vitamin K and the prevention of fractures: systematic review and meta-analysis of randomized controlled trials. Arch Intern Med 2006;166:1256–61.doi:10.1001/archinte.166.12.1256
      OpenUrlCrossRefPubMedWeb of Science
    19. 19.↵
      2. Torgerson DJ
      . Caution to readers about systematic review on vitamin K and prevention of fractures that included problematic trials. JAMA Intern Med 2018;178:863–4.doi:10.1001/jamainternmed.2018.1127
      OpenUrl
    20. 20.↵
      Caution of conclusions of a meta-analysis including problematic trials. JAMA Intern Med 2018;178:875–6.doi:10.1001/jamainternmed.2018.1140
      OpenUrl
    21. 21.↵
      2. Zhang W ,
      3. Zhu C ,
      4. Sun M , et al
      . Efficacy of bisphosphonates against hip fracture in elderly patients with stroke and Parkinson diseases: meta-analysis of randomized controlled trials. J Stroke Cerebrovasc Dis 2014;23:2714–24.doi:10.1016/j.jstrokecerebrovasdis.2014.06.022
      OpenUrl
    22. 22.↵
      2. Zhao Y ,
      3. Shen L ,
      4. Ji H-F
      . Alzheimer's disease and risk of hip fracture: a meta-analysis study. ScientificWorldJournal 2012;2012:1–5.doi:10.1100/2012/872173
      OpenUrlCrossRefPubMed
    23. 23.↵
      2. Orimo H ,
      3. Nakamura T ,
      4. Hosoi T , et al
      . Japanese 2011 guidelines for prevention and treatment of osteoporosis--executive summary. Arch Osteoporos 2012;7:3–20.doi:10.1007/s11657-012-0109-9
      OpenUrlCrossRefPubMed
    24. 24.↵
      2. Dai Z ,
      3. Koh W-P
      . B-vitamins and bone health--a review of the current evidence. Nutrients 2015;7:3322–46.doi:10.3390/nu7053322
      OpenUrl
    25. 25.↵
      2. Richy F ,
      3. Ethgen O ,
      4. Bruyere O , et al
      . Efficacy of alphacalcidol and calcitriol in primary and corticosteroid-induced osteoporosis: a meta-analysis of their effects on bone mineral density and fracture rate. Osteoporos Int 2004;15:301–10.doi:10.1007/s00198-003-1570-5
      OpenUrlCrossRefPubMedWeb of Science
    26. 26.↵
      2. Richy F ,
      3. Schacht E ,
      4. Bruyere O , et al
      . Vitamin D analogs versus native vitamin D in preventing bone loss and osteoporosis-related fractures: a comparative meta-analysis. Calcif Tissue Int 2005;76:176–86.doi:10.1007/s00223-004-0005-4
      OpenUrlCrossRefPubMedWeb of Science
    27. 27.↵
      2. Murad MH ,
      3. Drake MT ,
      4. Mullan RJ , et al
      . Clinical review. Comparative effectiveness of drug treatments to prevent fragility fractures: a systematic review and network meta-analysis. J Clin Endocrinol Metab 2012;97:1871–80.doi:10.1210/jc.2011-3060
      OpenUrlCrossRefPubMedWeb of Science
    28. 28.↵
      2. Yang J ,
      3. Hu X ,
      4. Zhang Q , et al
      . Homocysteine level and risk of fracture: a meta-analysis and systematic review. Bone 2012;51:376–82.doi:10.1016/j.bone.2012.05.024
      OpenUrlCrossRefPubMedWeb of Science
    29. 29.↵
      2. McCarus DC
      . Fracture prevention in postmenopausal osteoporosis: a review of treatment options. Obstet Gynecol Surv 2006;61:39–50.doi:10.1097/01.ogx.0000197807.08697.06
      OpenUrlPubMed
    30. 30.↵
      2. Gillespie LD ,
      3. Robertson MC ,
      4. Gillespie WJ , et al
      . Interventions for preventing falls in older people living in the community. Cochrane Database Syst Rev 2012;85.doi:10.1002/14651858.CD007146.pub3
    31. 31.↵
      2. Batchelor F ,
      3. Hill K ,
      4. Mackintosh S , et al
      . What works in falls prevention after stroke?: a systematic review and meta-analysis. Stroke 2010;41:1715–22.doi:10.1161/STROKEAHA.109.570390
      OpenUrlAbstract/FREE Full Text
    32. 32.↵
      2. Herrmann M ,
      3. Peter Schmidt J ,
      4. Umanskaya N , et al
      . The role of hyperhomocysteinemia as well as folate, vitamin B(6) and B(12) deficiencies in osteoporosis: a systematic review. Clin Chem Lab Med 2007;45:1621–32.doi:10.1515/CCLM.2007.362
      OpenUrlCrossRefPubMed
    33. 33.↵
      2. Peterson AL
      . A review of vitamin D and Parkinson's disease. Maturitas 2014;78:40–4.doi:10.1016/j.maturitas.2014.02.012
      OpenUrlCrossRefPubMedWeb of Science
    34. 34.↵
      2. Binks S ,
      3. Dobson R
      . Risk factors, epidemiology and treatment strategies for metabolic bone disease in patients with neurological disease. Curr Osteoporos Rep 2016;14:199–210.doi:10.1007/s11914-016-0320-5
      OpenUrl
    35. 35.↵
      2. Alibhai SMH ,
      3. Gogov S ,
      4. Allibhai Z
      . Long-term side effects of androgen deprivation therapy in men with non-metastatic prostate cancer: a systematic literature review. Crit Rev Oncol Hematol 2006;60:201–15.doi:10.1016/j.critrevonc.2006.06.006
      OpenUrlCrossRefPubMedWeb of Science
    36. 36.↵
      2. Carda S ,
      3. Cisari C ,
      4. Invernizzi M , et al
      . Osteoporosis after stroke: a review of the causes and potential treatments. Cerebrovasc Dis 2009;28:191–200.doi:10.1159/000226578
      OpenUrlPubMed
    37. 37.↵
      2. Simpson JSA ,
      3. Crawford SG ,
      4. Goldstein ET , et al
      . Systematic review of safety and tolerability of a complex micronutrient formula used in mental health. BMC Psychiatry 2011;11:62.doi:10.1186/1471-244X-11-62
    38. 38.↵
      2. Bjelakovic G ,
      3. Gluud LL ,
      4. Nikolova D , et al
      . Vitamin D supplementation for prevention of mortality in adults. Cochrane Database Syst Rev 2014;84.doi:10.1002/14651858.CD007470.pub3
    39. 39.↵
      2. Lv Z ,
      3. Qi H ,
      4. Wang L , et al
      . Vitamin D status and Parkinson's disease: a systematic review and meta-analysis. Neurol Sci 2014;35:1723–30.doi:10.1007/s10072-014-1821-6
      OpenUrl
    40. 40.↵
      2. Marsden J ,
      3. Gibson LM ,
      4. Lightbody CE , et al
      . Can early onset bone loss be effectively managed in post-stroke patients? an integrative review of the evidence. Age Ageing 2008;37:142–50.doi:10.1093/ageing/afm198
      OpenUrlCrossRefPubMedWeb of Science
    41. 41.↵
      2. Rucklidge JJ ,
      3. Kaplan BJ
      . Broad-spectrum micronutrient formulas for the treatment of psychiatric symptoms: a systematic review. Expert Rev Neurother 2013;13:49–73.doi:10.1586/ern.12.143
      OpenUrlCrossRefPubMed
    42. 42.↵
      2. Salari P ,
      3. Larijani B ,
      4. Abdollahi M
      . Association of hyperhomocysteinemia with osteoporosis: a systematic review. Therapy 2008;5:215–22.doi:10.2217/14750708.5.2.215
      OpenUrl
    43. 43.↵
      2. Zhao Y ,
      3. Sun Y ,
      4. Ji H-F , et al
      . Vitamin D levels in Alzheimer's and Parkinson's diseases: a meta-analysis. Nutrition 2013;29:828–32.doi:10.1016/j.nut.2012.11.018
      OpenUrl
    44. 44.↵
      2. Avenell A ,
      3. Mak JCS ,
      4. O'Connell D , et al
      . Vitamin D and vitamin D analogues for preventing fractures in post-menopausal women and older men. Cochrane Database Syst Rev 2014;39(4).doi:10.1002/14651858.CD000227.pub4
    45. 45.↵
      2. Cameron ID ,
      3. Gillespie LD ,
      4. Robertson MC , et al
      . Interventions for preventing falls in older people in care facilities and hospitals. Cochrane Database Syst Rev 2012;67.doi:10.1002/14651858.CD005465.pub3
    46. 46.↵
      2. Latham NK ,
      3. Anderson CS ,
      4. Reid IR
      . Effects of vitamin D supplementation on strength, physical performance, and falls in older persons: a systematic review. J Am Geriatr Soc 2003;51:1219–26.doi:10.1046/j.1532-5415.2003.51405.x
      OpenUrlCrossRefPubMedWeb of Science
    47. 47.↵
      2. Verheyden GSAF ,
      3. Weerdesteyn V ,
      4. Pickering RM , et al
      . Interventions for preventing falls in people after stroke. Cochrane Database Syst Rev 2013;26.doi:10.1002/14651858.CD008728.pub2
    48. 48.↵
      2. Reid IR ,
      3. Mason B ,
      4. Horne A , et al
      . Randomized controlled trial of calcium in healthy older women. Am J Med 2006;119:777–85.doi:10.1016/j.amjmed.2006.02.038
      OpenUrlCrossRefPubMedWeb of Science
    49. 49.↵
      2. Kernan WN ,
      3. Ovbiagele B ,
      4. Black HR , et al
      . Guidelines for the prevention of stroke in patients with stroke and transient ischemic attack: a guideline for healthcare professionals from the American heart Association/American stroke association. Stroke 2014;45:2160–236.doi:10.1161/STR.0000000000000024
      OpenUrlAbstract/FREE Full Text
    50. 50.↵
      1. Healthcare Improvement Scotland
      . SIGN 142. management of osteoporosis and the prevention of fragility fractures. Scottish Intercollegiate guidelines network: Edinburgh; 2015.
    51. 51.↵
      2. MacLean C ,
      3. Alexander A ,
      4. Carter J
      . Comparative effectiveness of treatments to prevent fractures in men and women with low bone density or osteoporosis. Comparative effectiveness review number 12. Rockville, MA Agency for Healthcare Research and Quality; 2007. et al.
    52. 52.↵
      2. MacLean C ,
      3. Newberry S ,
      4. Maglione M , et al
      . Systematic review: comparative effectiveness of treatments to prevent fractures in men and women with low bone density or osteoporosis. Ann Intern Med 2008;148:197–213.doi:10.7326/0003-4819-148-3-200802050-00198
      OpenUrlCrossRefPubMedWeb of Science
    53. 53.↵
      2. Qaseem A ,
      3. Snow V ,
      4. Shekelle P , et al
      . Pharmacologic treatment of low bone density or osteoporosis to prevent fractures: a clinical practice guideline from the American College of physicians. Ann Intern Med 2008;149:404–15.doi:10.7326/0003-4819-149-6-200809160-00007
      OpenUrlCrossRefPubMedWeb of Science
    54. 54.↵
      2. Crandall CJ ,
      3. Newberry SJ ,
      4. Diamant A , et al
      . Treatment to prevent fractures in men and women with low bone density or osteoporosis: update of a 2007 report. Rockville, MA Agency for Healthcare Research and Quality; 2012.
    55. 55.↵
      2. Qaseem A ,
      3. Forciea MA ,
      4. McLean RM , et al
      . Treatment of low bone density or osteoporosis to prevent fractures in men and women: a clinical practice guideline update from the American College of physicians. Ann Intern Med 2017;166:818–39.doi:10.7326/M15-1361
      OpenUrl
    56. 56.↵
      2. Currie L
      . Fall and Injury Prevention. In: Patient safety and quality: an evidence-based handbook for nurses [Internet. Rockville, MD: Agency for Healthcare Research and Quality, 2008.
    57. 57.↵
      2. Michael YL ,
      3. Lin JS ,
      4. Whitlock EP , et al
      . Interventions to prevent falls in older adults. An updated systematic review. Evidence syntheses. Rockville, MA Agency for Healthcare Research and Quality; 2010.
    58. 58.↵
      2. Bauman WA ,
      3. Spungen AM ,
      4. Morrison N , et al
      . Effect of a vitamin D analog on leg bone mineral density in patients with chronic spinal cord injury. JRRD 2005;42:625–33.doi:10.1682/JRRD.2004.11.0145
      OpenUrl
    59. 59.↵
      2. Berendsen A ,
      3. Santoro A ,
      4. Pini E , et al
      . A parallel randomized trial on the effect of a healthful diet on inflammageing and its consequences in European elderly people: design of the NU-AGE dietary intervention study. Mech Ageing Dev 2013;134:523–30.doi:10.1016/j.mad.2013.10.002
      OpenUrlCrossRefPubMed
    60. 60.↵
      2. Binkley N ,
      3. Harke J ,
      4. Krueger D , et al
      . Vitamin K treatment reduces undercarboxylated osteocalcin but does not alter bone turnover, density, or geometry in healthy postmenopausal North American women. J Bone Miner Res 2009;24:983–91.doi:10.1359/jbmr.081254
      OpenUrlCrossRefPubMed
    61. 61.↵
      2. Emaus N ,
      3. Nguyen ND ,
      4. Almaas B , et al
      . Serum level of under-carboxylated osteocalcin and bone mineral density in early menopausal Norwegian women. Eur J Nutr 2013;52:49–55.doi:10.1007/s00394-011-0285-1
      OpenUrl
    62. 62.↵
      2. Grieger JA ,
      3. Nowson CA ,
      4. Jarman HF , et al
      . Multivitamin supplementation improves nutritional status and bone quality in aged care residents. Eur J Clin Nutr 2009;63:558–65.doi:10.1038/sj.ejcn.1602963
      OpenUrlCrossRefPubMed
    63. 63.↵
      2. Herrmann M ,
      3. Umanskaya N ,
      4. Traber L , et al
      . The effect of B-vitamins on biochemical bone turnover markers and bone mineral density in osteoporotic patients: a 1-year double blind placebo controlled trial. Clin Chem Lab Med 2007;45:1785–92.doi:10.1515/CCLM.2007.352
      OpenUrlPubMed
    64. 64.↵
      2. Rucklidge JJ ,
      3. Andridge R ,
      4. Gorman B , et al
      . Shaken but unstirred? effects of micronutrients on stress and trauma after an earthquake: RCT evidence comparing formulas and doses. Hum Psychopharmacol 2012;27:440–54.doi:10.1002/hup.2246
      OpenUrlCrossRefPubMed
    65. 65.↵
      2. van Wijngaarden JP ,
      3. Swart KMA ,
      4. Enneman AW , et al
      . Effect of daily vitamin B-12 and folic acid supplementation on fracture incidence in elderly individuals with an elevated plasma homocysteine concentration: B-PROOF, a randomized controlled trial. Am J Clin Nutr 2014;100:1578–86.doi:10.3945/ajcn.114.090043
      OpenUrlAbstract/FREE Full Text
    66. 66.↵
      2. Agrawal S ,
      3. Krueger DC ,
      4. Engelke JA , et al
      . Between-meal risedronate does not alter bone turnover in nursing home residents. J Am Geriatr Soc 2006;54:790–5.doi:10.1111/j.1532-5415.2006.00696.x
      OpenUrlPubMed
    67. 67.↵
      2. Gommans J ,
      3. Yi Q ,
      4. Eikelboom JW , et al
      . The effect of homocysteine-lowering with B-vitamins on osteoporotic fractures in patients with cerebrovascular disease: substudy of VITATOPS, a randomised placebo-controlled trial. BMC Geriatr 2013;13:88.doi:10.1186/1471-2318-13-88
    68. 68.↵
      2. Green TJ ,
      3. McMahon JA ,
      4. Skeaff CM , et al
      . Lowering homocysteine with B vitamins has no effect on biomarkers of bone turnover in older persons: a 2-y randomized controlled trial. Am J Clin Nutr 2007;85:460–4.doi:10.1093/ajcn/85.2.460
      OpenUrlAbstract/FREE Full Text
    69. 69.↵
      2. Sawka AM ,
      3. Ray JG ,
      4. Yi Q , et al
      . Randomized clinical trial of homocysteine level-lowering therapy and fractures. Arch Intern Med 2007;167:2136–9.doi:10.1001/archinte.167.19.2136
      OpenUrlCrossRefPubMed
    70. 70.↵
      2. Booth SL ,
      3. Dallal G ,
      4. Shea MK , et al
      . Effect of vitamin K supplementation on bone loss in elderly men and women. J Clin Endocrinol Metab 2008;93:1217–23.doi:10.1210/jc.2007-2490
      OpenUrlCrossRefPubMedWeb of Science
    71. 71.↵
      2. Emaus N ,
      3. Gjesdal CG ,
      4. Almås B , et al
      . Vitamin K2 supplementation does not influence bone loss in early menopausal women: a randomised double-blind placebo-controlled trial. Osteoporos Int 2010;21:1731–40.doi:10.1007/s00198-009-1126-4
      OpenUrlCrossRefPubMed
    72. 72.↵
      2. Herrmann W ,
      3. Kirsch SH ,
      4. Kruse V , et al
      . One year B and D vitamins supplementation improves metabolic bone markers. Clin Chem Lab Med 2013;51:639–47.doi:10.1515/cclm-2012-0599
      OpenUrlPubMed
    73. 73.↵
      2. Shiraki M ,
      3. Itabashi A
      . Short-term menatetrenone therapy increases gamma-carboxylation of osteocalcin with a moderate increase of bone turnover in postmenopausal osteoporosis: a randomized prospective study. J Bone Miner Metab 2009;27:333–40.doi:10.1007/s00774-008-0034-6
      OpenUrlCrossRefPubMed
    74. 74.↵
      2. Marret E ,
      3. Elia N ,
      4. Dahl JB , et al
      . Susceptibility to fraud in systematic reviews: lessons from the Reuben case. Anesthesiology 2009;111:1279–89.doi:10.1097/ALN.0b013e3181c14c3d
      OpenUrlCrossRefPubMedWeb of Science
    75. 75.↵
      2. Moore RA ,
      3. Derry S ,
      4. McQuay HJ
      . Fraud or flawed: adverse impact of fabricated or poor quality research. Anaesthesia 2010;65:327–30.doi:10.1111/j.1365-2044.2010.06295.x
      OpenUrlCrossRefPubMedWeb of Science
    76. 76.↵
      2. Sox HC ,
      3. Rennie D
      . Research misconduct, retraction, and cleansing the medical literature: lessons from the Poehlman case. Ann Intern Med 2006;144:609–13.doi:10.7326/0003-4819-144-8-200604180-00123
      OpenUrlCrossRefPubMedWeb of Science
    77. 77.↵
      1. Retraction Watch
      . What should you do if a paper you’ve cited is later retracted? Available: http://retractionwatch.com/2016/11/01/what-should-you-do-if-a-paper-youve-cited-is-later-retracted/ [Accessed 23 Nov 2018].
    78. 78.↵
      2. Barbour V ,
      3. Bloom T ,
      4. Lin J , et al
      . Amending published articles: time to rethink retractions and corrections? F1000Research 1960;2017.
    79. 79.↵
      2. Habib AS ,
      3. Gan TJ
      . Scientific fraud: impact of Fujii's data on our current knowledge and practice for the management of postoperative nausea and vomiting. Anesth Analg 2013;116:520–2.doi:10.1213/ANE.0b013e31827ab7d8
      OpenUrlCrossRefPubMed
    80. 80.↵
      2. Wiedermann CJ
      . Inaction over retractions of identified fraudulent publications: ongoing weakness in the system of scientific self-correction. Account Res 2018;25:239–53.doi:10.1080/08989621.2018.1450143
      OpenUrl
    81. 81.↵
      2. Bornemann-Cimenti H ,
      3. Szilagyi IS ,
      4. Sandner-Kiesling A
      . Perpetuation of retracted publications using the example of the Scott S. Reuben case: incidences, reasons and possible improvements. Sci Eng Ethics 2016;22:1063–72.doi:10.1007/s11948-015-9680-y
      OpenUrl
    82. 82.↵
      2. Bar-Ilan J ,
      3. Halevi G
      . Temporal characteristics of retracted articles. Scientometrics 2018;116:1771–83.doi:10.1007/s11192-018-2802-y
      OpenUrl
    83. 83.↵
      2. Mott A ,
      3. Fairhurst C ,
      4. Torgerson D
      . Assessing the impact of retraction on the citation of randomized controlled trial reports: an interrupted time-series analysis. J Health Serv Res Policy 2019;24:44–51.doi:10.1177/1355819618797965
      OpenUrl
    84. 84.↵
      2. White PF ,
      3. Kehlet H ,
      4. Liu S
      . Perioperative analgesia: what do we still know? Anesth Analg 2009;108:1364–7.doi:10.1213/ane.0b013e3181a16835
      OpenUrlCrossRefPubMedWeb of Science
    85. 85.↵
      2. Clarke CE
      . Efficacy of methyprednisolone pulse therapy on neuroleptic malignant syndrome in Parkinson's disease. J Neurol Neurosurg Psychiatry 2004;75:510–1.
      OpenUrlPubMed
    86. 86.↵
      2. Poole KES ,
      3. Warburton EA ,
      4. Reeve J
      . Risedronate therapy for prevention of hip fracture after stroke in elderly women. Neurology 2005;65:1513–4.doi:10.1212/WNL.65.9.1513
      OpenUrl
    87. 87.↵
      2. Caffrey N ,
      3. Casey M ,
      4. Walsh B
      . Substandard studies or substandard "standard-of-care". J Bone Miner Res 2006;21:491.doi:10.1359/JBMR.051208
    88. 88.↵
      2. Halbekath JM ,
      3. Schenk S ,
      4. von Maxen A , et al
      . Risedronate for the prevention of hip fractures: concern about validity of trials. Arch Intern Med 2007;167:513–4.doi:10.1001/archinte.167.5.513-b
      OpenUrlCrossRefPubMed
    89. 89.↵
      2. Gøtzsche PC ,
      3. Hróbjartsson A ,
      4. Marić K , et al
      . Data extraction errors in meta-analyses that use standardized mean differences. JAMA 2007;298:430–7.doi:10.1001/jama.298.4.430
      OpenUrlCrossRefPubMedWeb of Science
    90. 90.↵
      2. Mistry V ,
      3. Grey A ,
      4. Bolland MJ
      . Publication rates after the first retraction for biomedical researchers with multiple retracted publications. Account Res 2019;26:277–87.doi:10.1080/08989621.2019.1612244
      OpenUrl
    91. 91.↵
      2. van der Vet PE ,
      3. Nijveen H
      . Propagation of errors in citation networks: a study involving the entire citation network of a widely cited paper published in, and later retracted from Nature. Res Integr Peer Rev 2016;1.doi:10.1186/s41073-016-0008-5
    92. 92.↵
      2. Fanelli D ,
      3. Moher D
      . What difference do retractions make? an estimate of the epistemic impact of retractions on recent meta-analyses. bioRxiv 2019.
    93. 93.↵
      2. Elia N ,
      3. Wager E ,
      4. Tramèr MR
      . Fate of articles that warranted retraction due to ethical concerns: a descriptive cross-sectional study. PLoS One 2014;9:e85846.doi:https://doi-org.ezproxy.u-pec.fr/10.1101/734137
    94. 94.↵
      1. Zotero
      . Retracted item notifications with retraction Watch integration. Available: https://www.zotero.org/blog/retracted-item-notifications/ [Accessed 7 Aug 2019].
    95. 95.↵
      2. Cheng YY ,
      3. Parulian N ,
      4. Hsiao TK , et al
      . ReTracker: actively and automatically matching retraction metadata in Zotero. 82nd Annual Meeting of the Association for Information Science & Technology, Melbourne, Australia, 2019. Available: http://jodischneider.com/pubs/asist2019retracker.pdf [Accessed 07 Aug 2019].

    Footnotes

    • Contributors AA, FS, AG, GG and MB conceived and designed the study. FS and AA collected data. FS, AA, AG and MB analysed data. FS and AA wrote the draft manuscript. AA, FS, AG, GG and MB revised and approved the final submission. AA is the guarantor of the paper.

    • Funding The Health Services Research Unit at the University of Aberdeen is funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates.

    • Competing interests All authors have completed the ICMJE unified disclosure form competing interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author). AG is a shareholder in Auckland Bone Density, a company that provides bone mineral density measurements. MJB and AG report grants from the Health Research Council of New Zealand during the conduct of this study. AA reports grants from NIHR during the conduct of this study. FS and GDG have no financial conflict of interest to declare.

    • Patient consent for publication Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement Data are available upon reasonable request.

    Linked Articles