Integration of academic and health education for the prevention of physical aggression and violence in young people: systematic review, narrative synthesis and intervention components analysis
================================================================================================================================================================================================

* G.J. Melendez-Torres
* Tara Tancred
* Adam Fletcher
* Rona Campbell
* James Thomas
* Christopher Bonell

## Abstract

**Objectives** To systematically review evidence on the effectiveness of interventions including integration of academic and health education for reducing physical aggression and violence, and describe the content of these interventions.

**Data sources** Between November and December 2015, we searched 19 databases and 32 websites and consulted key experts in the field. We updated our search in February 2018.

**Eligibility criteria** We included randomised trials of school-based interventions integrating academic and health education in students aged 4–18 and not targeted at health-related subpopulations (eg, learning or developmental difficulties). We included evaluations reporting a measure of interpersonal violence or aggression.

**Data extraction and analysis** Data were extracted independently in duplicate, interventions were analysed to understand similarities and differences and outcomes were narratively synthesised by key stage (KS).

**Results** We included 13 evaluations of 10 interventions reported in 20 papers. Interventions included either full or partial integration, incorporated a variety of domains beyond the classroom, and used literature, local development or linking of study skills and health promoting skills. Evidence was concentrated in KS2, with few evaluations in KS3 or KS4, and evaluations had few consistent effects; evaluations in KS3 and KS4 did not suggest effectiveness.

**Discussion** Integration of academic and health education may be a promising approach, but more evidence is needed. Future research should consider the ‘lifecourse’ aspects of these interventions; that is, do they have a longitudinal effect? Evaluations did not shed light on the value of different approaches to integration.

*   systematic review
*   violence
*   community child health

### Strengths and limitations of this study

*   We used an exhaustive search including 19 databases and 32 websites.

*   We used an innovative method to describe key components in this class of interventions.

*   However, it was challenging to identify studies for inclusion.

*   Meta-analysis was not possible because of the diversity of outcomes and raters.

## Introduction 

Violence among young people is a public health priority due to its prevalence and harm to young people and the wider society.1 2 One UK study found that 10% of young people aged 11–12 reported carrying a weapon and 8% admitted attacking someone with intent to hurt them seriously.3 By age 15–16, 24% of students reported they have carried a weapon and 19% reported attacking someone with the intention to hurt them seriously.3 Early aggression and antisocial behaviour are strongly linked to adult violent behaviour.4 5 

School-based health education can be effective in reducing violence.6–8 However, school-based health education is increasingly marginal in many high-income countries, partly because of schools increasing focus on attainment-based performance metrics. In England specifically, health education is not a statutory subject,9–11 and school inspectors have a limited focus on how schools promote student health.12 

One way to avoid such marginalisation is to integrate health education into academic lessons. For example, health-related content can be seamlessly integrated into existing academic lessons or discrete additional health education lessons can also include academic learning elements. This strategy may bring other benefits because: larger ‘doses’ may be delivered; students may be less resistant to health messages weaved into other subjects; and lessons in different subjects may reinforce each other.13 14 Conversely, those teaching academic subjects may be uninterested or unqualified to teach health topics. Though theories of change in this class of interventions are diffuse, one important way in which they could be effective is by promoting developmental cascades involving the interplay of cognitive and non-cognitive skills.15 16 Interventions integrating academic and health education could address violence by developing: social and emotional skills such as self-awareness, self-regulation, motivation, empathy and communication17; healthier social support or norms among students15 18 19; knowledge of the costs20 and consequences21 of substance use; media literacy skills to critique harmful media messages; and modifying students’ social norms about antisocial behaviours.13 20 22–24 Our work synthesising the theories of change underlying these interventions (Tancred *et al*, in press) identified that interventions aimed to integrate and thus erode boundaries between academic and health education, between students and teachers (so that relationships were improved and teachers might function more effectively as behavioural role models) and between classrooms and schools and schools and families (so that violence prevention messages communicated in classrooms might be reinforced by messaging in other settings).

Despite policy interest in these interventions, they have not previously been the subject of a specific systematic review. Previous systematic reviews have focused on socioemotional learning interventions or school-based interventions generally,6–8 without considering interventions that specifically integrate with academic lessons as defined above. Our focus on violence is informed by preliminary consultation, scoping work and logic model development suggesting that violence is an outcome especially amenable to these interventions. In the present review, we examined the characteristics of interventions that integrate academic and health education to prevent violence, and synthesised evidence for their effectiveness. That is, our research questions were: what are the overarching features relevant to integration of interventions that integrate academic and health education, and are these interventions effective at different key stages (KS) in reducing physical aggression and violence?

## Methods

This review was part of a larger evidence synthesis project on theories of change, process evaluations and outcome evaluations of integration of academic and health education for substance use and violence. We registered the protocol for this review on PROSPERO (CRD42015026464, [https://www.crd.york.ac.uk/prospero/](https://www.crd.york.ac.uk/prospero/)), and it is enclosed as online supplementary file 1.

### Supplementary file 2

[[bmjopen-2017-020793-SP2.pdf]](pending:yes) 

### Inclusion and exclusion

Studies were included regardless of publication date or language. We included randomised controlled trials of interventions integrating academic and health education, the former defined as specific academic subjects or general study skills. We defined education as ‘health education’ seeking to improve the health and well-being of students (including social and emotional learning and other forms of violence prevention). We included school-based interventions that seamlessly incorporated health education into existing academic lessons and interventions that provided discrete health education lessons with additional academic components. Interventions could be delivered by teachers or other school staff such as teaching assistants, but may also have been delivered by external providers, for example, from the health, voluntary or youth service sectors. We did not include interventions solely addressing social conduct in the classroom; relationships with peers or staff; attitudes to education, school or teachers; or aspirations and life goals. Our definition also excluded interventions which: were delivered in mainstream subject lessons but did not aim to integrate health and academic education; trained teachers in classroom management without student curriculum components; or were delivered exclusively outside of classrooms, as these did not seek to integrate academic and health education. Interventions focusing on targeted health-related subpopulations (eg, children with cognitive disabilities) were excluded as we were interested in universal interventions.

For this review, we focus on violence outcomes, defined as the perpetration or victimisation of physical violence including convictions for violent crime. While we preferred direct measures of physically violent and physically aggressive behaviours, we included outcomes that were a composite of physical and non-physical (eg, verbal or emotional) interpersonal violence, but excluded composite measures that also included items not focused on interpersonal violence, such as damage to property.

### Search strategy

In our original search, undertaken between November and December 2015, we searched 19 databases and 32 websites, and contacted subject experts (see online supplementary file 2 for full details). We subsequently updated our search in February 2018 using PsycINFO and CENTRAL, as all of our original study hits were recovered from these databases.

### Supplementary file 1

[[bmjopen-2017-020793-SP1.pdf]](pending:yes) 

### Study selection

Pairs of researchers double-screened titles and abstracts in sets of 50 references until 90% agreement was reached, with disagreements discussed at every stage. Subsequently, single reviewers screened each reference. We located the full texts of remaining references and undertook similar pairwise calibration with disagreements discussed, followed by single screening. Reports were translated into English where necessary. Using an existing tool,25 we extracted data independently in duplicate from included studies and assessed trials for risk of bias using a modified version of the Cochrane assessment tool.26 Authors were contacted where study data were missing.

### Synthesis methods

We undertook an intervention components analysis.27 This was undertaken inductively by one researcher and audited by two other researchers, and used intervention descriptions to draw out similarities and differences in intervention design using an iterative method. Intervention descriptions were read and reread and then coded manually. The goal of this analysis was to use a set of descriptors to characterise aspects of the integration of academic and health education in the intervention. Intervention descriptions were rarely detailed enough to permit ‘deep’ engagement with the specific content of the interventions provided in included evaluations. The intervention components analysis identified overarching domains that accounted for similarities and differences between interventions in their integration of academic and health education, and developed within each domain a set of overlapping categories that described these similarities and differences. Finally, we synthesised outcomes narratively due to the heterogeneity in included outcome measurement. We categorised the timing of intervention effect by period of schooling, defined in terms of English schools’ KS system. KS1 includes school years 1–2 (age 5–7 years), KS2 includes years 3–6 (age 7–11 years), KS3 includes years 7–9 (age 11–14 years), KS4 includes years 10–11 (age 14–16 years) and KS5 includes years 12–13 (age 16–18 years).

We could not formally assess publication bias because heterogeneity in outcome measurement precluded meta-analysis.

### Patient and public involvement

Because this review focused on public health interventions that were generally preventive in nature, patients were not involved per se. However, stakeholders were extensively consulted in the development of research questions and in assessing the implications of the findings. In addition, findings were disseminated via stakeholder events, and a series of one-to-one consultations took place to ensure the relevance and salience of study findings.

## Results

In our original search, we found and screened 76 979 references, of which we retained 702 for full-text screening and were able to assess 690. Of 62 relevant reports included in the overall project, 10 evaluations of eight interventions were reported in 14 papers that considered violence and are reported in this review. Our update search yielded 2355 references, of which we retained 41 for full-text screening and included six papers reporting three evaluations (figure 1). This yielded a total of 13 evaluations reported in 20 papers.

![Figure 1](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/8/9/e020793/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/8/9/e020793/F1)

Figure 1 
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart. 

### Included studies and their quality

All trials randomised schools except the Bullying Literature Project, which randomised classrooms (table 1). All evaluations were conducted in the USA, except for Gatehouse,28 which was an Australian study, and Learning to Read in a Healing Classroom,29 30 which took place in the Democratic Republic of the Congo. All control arms consisted of education-as-usual or waitlist controls, though Second Step31–33 offered a brief antibullying intervention with low take-up.

View this table:
[Table 1](http://bmjopen.bmj.com/content/8/9/e020793/T1)

Table 1 
Characteristics of included studies

Interventions were diverse and are summarised below in the intervention components analysis. Only two interventions (Bullying Literature Project,34 Youth Matters35) were wholly delivered by external staff. Several (Gatehouse,28 Positive Action,36 Steps to Respect37) linked classroom-based delivery to school-level work to support and reinforce implementation. Promoting Alternative Thinking Strategies (PATHS)38 and Reading, Writing, Respect and Reconciliation (4Rs)19 also emphasised teachers’ professional development.

Evaluation quality varied (table 2). Appraisal was hampered by poor reporting of some aspects of trial methods. Only four studies reported evidence of low risk of bias for random generation of allocation sequence; the remainder were unclear. Only one study reported information on concealed allocation. In Linking the Interests of Families and Teachers (LIFT),39 outcome assessors were blinded, resulting in low risk of bias in this domain, but all other interventions were of unclear risk of bias. All interventions included reasonably complete outcome data, and in only one evaluation did unit of analysis issues pose a risk of bias. In some studies such as Steps to Respect, follow-up was shorter than intervention length. Evaluations also differed in size, ranging from 7 classrooms to 63 schools.

View this table:
[Table 2](http://bmjopen.bmj.com/content/8/9/e020793/T2)

Table 2 
Appraisal of included studies

### Intervention components analysis

This identified four themes describing included interventions: approach to integration, position of integration, degree of integration and point of integration. Included interventions are described in table 1, and the components analysis is summarised in table 3.

View this table:
[Table 3](http://bmjopen.bmj.com/content/8/9/e020793/T3)

Table 3 
Key themes in the intervention components analysis

#### Approach to integration

Interventions approached the rationale for and strategy of integration in different and overlapping ways. These overlapped across interventions, but were not mutually exclusive, and described the types of academic foci that interventions used to integrate academic and health education. Several (4Rs, Bullying Literature Project, Steps to Respect, Youth Matters) focused on *literature* as a focus for integration, using children’s books as a prompt for social–emotional learning. These interventions targeted language arts or literacy lessons as an opportunity to provoke discussion, role play and model positive strategies to avoid violence. Gatehouse explicitly used a ‘critical literacy’ approach to inspire reflection on programme lessons in English classes. Another approach to integration emphasised *local development*, where interventions supported teachers to link health education across academic subjects in each school in a ‘local’ fashion. For example, in PATHS, teachers received suggestions on how to integrate programme learning across English, history and social studies lessons, while in Second Step, this was an encouraged aspect of classroom delivery. In both cases, teachers received guidance and support to integrate health education messages into academic education, but were given substantial latitude to determine how and when to do this in the school day. A third approach was *linking to developmental concerns*, emphasising not so much the comprehensive integration of academic and health education but rather the inter-relationships between academic success and broader development, health and well-being. These interventions viewed academic education through a ‘health’ lens, in addition to viewing health education through an ‘academic’ lens. From a conceptual perspective, this meant that the inter-relationships between academic achievement, and student health and well-being were emphasised in theories of change. From a practical perspective, this meant that interventions paired activities such as study skills lessons with social–emotional learning (eg, in PATHS). For example, the theory of change underlying Gatehouse related to the creation of healthy social milieus in schools that would also support academic attainment; practically, this manifested as enhancement of academic lessons to improve interpersonal skills and emotional regulation. Similarly, Positive Action tied together individual student attainment with student health and well-being in their theory of change, with lessons focused on problem-solving and goal setting, among other topics.

#### Domains of integration

Some interventions (4Rs, Bullying Literature Project and Youth Matters) were exclusively *classroom-focused* while others (Gatehouse, Steps to Respect) used *classroom and whole-school* strategies to reinforce and extend learning. For example, Gatehouse involved school implementation support teams, while Steps to Respect deployed a school-wide ‘policy team’ to revise and develop antibullying policies. Other interventions, (PATHS, Positive Action) used *classroom, whole-school environment and external domain* (parent information) strategies consistent with the health-promoting schools approach promulgated particularly by the WHO, which in the USA is known as the Comprehensive School Health Programme model.40 

#### Degree of integration

In some interventions, health education was fully integrated (woven seamlessly) into everyday academic lessons (Gatehouse, 4Rs, Youth Matters), while in partially integrated interventions, health education involved distinct lessons, although also covering academic learning (Positive Action).

#### Timing of integration

Most interventions were multiyear, though two involved only 1 school year (LIFT, Bullying Literature Project).

### Intervention effects

Perpetration measures included bullying (physical or physical/verbal), aggression against peers and others and violent behaviours including injuring others. Measures involved different raters, including students, teachers and observers. Victimisation measures ranged from physical violence specifically to interpersonal aggression more generally. Heterogeneity of definition, measurement and form of effect sizes precluded meta-analysis. No included studies described effects for KS1 or KS5. Measures and corresponding effect estimates are included in table 4.

View this table:
[Table 4](http://bmjopen.bmj.com/content/8/9/e020793/T4)

Table 4 
Measures used in included studies and effect estimates

### Violence perpetration: KS2

Across the 10 evaluations reporting outcomes in this KS, effects were inconsistent, including within studies by rater.

In LIFT,39 effects at the end of the first intervention year on observed physical aggression in the playground were similar for students with different levels of baseline aggression (*d*=−0.14 at mean, 1 SD and 2 SD above the preintervention mean); these findings being described as ‘statistically significant’. However, after the first intervention year of 4Rs,19 there were no effects on teacher-reported aggression (regression-estimated *b*=0.02, SE=0.05, based on a 1–4 scale). After the second intervention year,15 there were effects on teacher-reported student aggression (*d*=−0.21, p<0.05). The Bullying Literature Project also reported no effects on physical aggression rated by teachers for individual students (IG [intervention group]: M=1.12, SD=0.47, n=95 vs CG [control group]: 1.19, SD=0.47, n=55; p=0.67) or student self-reports (M=1.20, SD=0.44, n=90 vs M=1.14, SD=0.36, n=42; p=0.84) at 1 week postintervention.34 This finding was the same in the Bullying Literature Project—Moral Disengagement version (F(1, 80)=0.83, p=0.431), though only combined student-reported physical and emotional bullying estimates were available.41 

Findings for Steps to Respect differed by type of rater. At the end of the first intervention year, the first evaluation of Steps to Respect37 reported evidence of decreased bullying based on playground observation (*F*(91.3)=5.02, p<0.01) but not direct aggression based on student report (*F*(68.7)=2.05, p>0.05). The second evaluation of Steps to Respect42 revealed a similar pattern. While teacher reports of physical bullying perpetration were less in intervention schools than in control schools at the end of the first intervention year (OR=0.61, *t*(29)=−3.12, p<0.01), student reports suggested no difference between schools on bullying perpetration (*t*(29)=−1.06). Moreover, in PATHS,38 small positive effects of the intervention on student-reported aggression at the end of the first intervention year (*d*=−0.048, 95% CI −0.189 to 0.092) and at the start (*d* =−0.064, 95% CI −0.205 to 0.076) and end (*d* =−0.048, 95% CI −0.188 to 0.093) of the second intervention year gave way to a small deleterious intervention effect at the end of the third year (*d*=0.082, 95% CI −0.060 to 0.224). Opposite effects were found on teacher-reported aggression, with initially small, negative intervention effects at the end of the first (*d*=0.036, 95% CI −0.105 to 0.178) and start of the second intervention year (*d*=0.035, 95% CI −0.107 to 0.178) but progressively greater effects at the end of the second (*d*=−0.005, 95% CI −0.146 to 0.136) and the third (*d*=−0.199, 95% CI −0.338 to –0.060) intervention years.

In contrast, two evaluations showed consistently positive results across different measures. In Positive Action Chicago,36 students reported lower counts of bullying behaviours (incidence rate ratio (IRR)=0.59, 95% CI 0.37 to 0.92) and of serious violence-related behaviours, including cutting or stabbing someone on purpose (IRR=0.63, 95% CI 0.45, 0.88). Findings from Positive Action Hawaii43 were similar for student-reported violent behaviours (IRR=0.42, 90% CI 0.24 to 0.73) and teacher-reported violent behaviours (IRR=0.54, 90% CI 0.30, 0.77). For students in the fourth or fifth intervention year, intervention recipients were less likely to report cutting or stabbing someone (OR=0.29, 90% CI 0.16 to 0.52) or shooting someone (OR=0.24, 90% CI 0.14, 0.40). Teachers were less likely to report that students hurt others (OR=0.61, 90% CI 0.38, 0.97) or got into lots of fights (OR=0.63, 90% CI 0.47, 0.84).

However, in Youth Matters,35 students in intervention schools were not less likely to report bullying perpetration (OR=0.85, 95% CI 0.29 to 1.47, p=0.585) after the second intervention year. Evaluators explored use of latent class analyses to classify intervention recipients as victims, bullies or bully–victims. Proportions of intervention and control recipients classified as bullies or bully–victims were not significantly different by study arm at the end of the first (IG: 21%, n=356 vs CG: 22%, n=392) or second (19%, n=244 vs 23%, n=293) intervention years.44 

### Violence perpetration: KS3

The three evaluations examining violence perpetration outcomes in KS3 had dissimilar results. At the end of the sixth intervention year of Positive Action Chicago,45 students receiving the intervention reported lower counts of violence-related behaviours than no treatment controls (IRR=0.38, 95% CI 0.18 to 0.81; equivalent to *d*=−0.54). Students also reported fewer bullying behaviours (*d*=−0.39), and parents reported that their children engaged in fewer bullying behaviours (*d*=−0.31). Significance values for these estimates were not presented, but both were supported by significant condition by time interactions in multilevel models, indicating that the intervention group showed an improved trajectory over time as compared with the control group. In contrast, after the third year from baseline in Youth Matters,44 proportions of students were not different in the collective bully and bully–victim groups (both groups 16%; IG n=283, CG n=289). Findings for Second Step were reported at the end of the first, second and third years of intervention. At the end of the first school year, students in intervention schools had decreased odds of physical aggression (OR=0.70, p<0.05) but not sexual harassment and sexual violence perpetration (OR=1.04, p>0.05).33 These findings did not hold to the end of the second school year for physical aggression (OR=0.80, 95% CI 0.59 to 1.08), but sexual harassment and sexual violence perpetration was significantly reduced in intervention schools in Illinois (OR=0.72, 95% CI 0.54, 0.95) but not Kansas (OR=0.99, 95% CI 0.71, 1.48).32 At the end of the third school year, there were no direct effects of Second Step on sexual harassment perpetration (β=0.005, SE=0.012); findings for physical aggression were not available.31 

### Violence victimisation: KS2

While the seven evaluations reporting outcomes in this KS were similar in follow-up period, they did not point to a clear effect. Students receiving the ‘original’ Bullying Literature Project were not different from their peers in physical victimisation by teacher report on individual students (IG: M=1.04, SD=0.23, n=95 vs CG: 1.04, SD=0.21, n=55; P=0.39) or student self-report (M=1.35, SD=0.54, n=90 vs M=1.43, SD=0.66, n=42; P=0.57) 1 week postintervention.34 However, students receiving the Bullying Literature Project—Moral Disengagement version did report decrease in victimisation (both physical and emotional combined) after the intervention (IG: M=1.76, SD=0.81 to M=1.60, SD=0.66, n=42 vs CG: M=1.23, SD=0.38 to M=1.38, SD=0.53, n=42), with a significant time-by-treatment interaction in an analysis of variance (F(1, 80)=7.42, P=0.047).41 PATHS measured student-reported victimisation using standardised mean differences, and found small, non-significant increases relative to the control arm at: the end of the first intervention year (*d*=0.044, 95% CI −0.098 to 0.185); the start (*d*=0.074, 95% CI −0.067, 0.216) and end (*d*=0.092, 95% CI −0.050, 0.234) of the second year; and the end of the third year (*d*=0.089, 95% CI −0.053, 0.231) of intervention implementation.38 Steps to Respect, evaluated in two different trials, also found no differences in student-reported bullying victimisation at the end of the first intervention year in the first (IG: M=0.80, SD=1.51 vs CG: M=0.86, SD=1.44; *F*<1)37 or second trial (M=2.11, SD=1.03 vs M=2.18, SD=1.06; *t*(29)=-1.15).42 The first trial included playground observation at the end of the first intervention year, which was suggestive of lower levels in bullying victimisation, though these differences were marginally non-significant (M=0.9, SD=0.82 vs M=1.01, SD=0.83; *F*(72.4)=3.74, p<0.10).37 Learning to Read in a Healing Classroom examined relational and physical victimisation after 1 year of intervention implementation and found no significant effect of the intervention (weighted *d*=−0.01, SE=0.06).29 30 Finally, Youth Matters examined bullying victimisation through continuous and dichotomous measures. At the end of the second intervention year, the difference in log-transformed continuous scores suggested a decrease (difference=−0.171, SE=0.083, p=0.049), as did the difference in dichotomous scores (OR=0.61, p=0.098).35 However, a latent class analysis that sought to describe transitions into, and out of, bullying victimisation did not suggest a difference between groups at this point.44 

### Violence victimisation: KS3 and KS4

Intervention evaluations reporting violence victimisation outcomes in KS3 (Youth Matters,44 46 Second Step32 33 and Gatehouse28) and KS4 (Gatehouse28) suggested no evidence of effectiveness. In Youth Matters, differences in the log-transformed scores for bullying victimisation suggested a decrease in victimisation in intervention recipients as compared with controls, but this difference was not significant (regression-estimated difference=−0.123, SE=0.068, p=0.08).46 However, at the end of the third intervention year, fewer students in the intervention than control group were members of the victim or bully–victim classes (36%, n=283 vs 45%, n=289).44 Based on our own χ2 test, this difference was significant (p=0.029). In Second Step, neither peer victimisation (OR=1.01, p>0.5) nor sexual harassment and violence victimisation (OR=1.01, p>0.05) were different between students in intervention schools and control schools after the first intervention year.33 This remained the case at the end of the second intervention year (peer victimisation: OR=0.94, 95% CI 0.75, 1.18); sexual victimisation: OR=0.91, 95% CI 0.72, 1.15).32 Gatehouse,28 which was implemented from year 9, found no evidence of a change in bullying victimisation at the end of the first (OR=1.03, 95% CI 0.86, 1.26)), second (OR=1.03, 95% CI 0.78, 1.34) or third (OR=0.88, 95% CI 0.68, 1.13) intervention years, which corresponded to the first 2 years of KS4.

## Discussion

While the integration of academic and health education remains a promising model for the delivery of school-based health education, randomised evaluations were variable in quality and did not consistently report evidence of effectiveness in reducing violence victimisation or perpetration. Evidence was concentrated in KS2, with few evaluations in KS3 or KS4. Moreover, evidence was stronger in quantity and in quality for violence perpetration as compared with victimisation. Unfortunately, evaluations that measured perpetration did not always also measure victimisation, preventing a meaningful comparison of consistency of effects.

Few interventions showed consistent signals of effectiveness. Though a formal moderator analysis was not possible, certain intervention models appear more effective than others. Specifically, evaluations of Positive Action in both Chicago45 and Hawaii43 showed consistently positive results across diverse measures. This may reflect the involvement of the intervention developer, a factor often associated with improved intervention fidelity (although Positive Action was not unique in this respect among interventions included in our review). It may also reflect that Positive Action included classroom, whole-school and (in the Hawaii trial) external domain strategies delivered over multiple school years. Though Gatehouse28 was similar to Positive Action in its focus on multiple systems, Gatehouse targeted adolescents, whereas Positive Action was delivered from KS2 and also included work with parents. Another possible explanation for our results is that effects for these interventions may take time to emerge. This is plausible given the developmental focus of many of these interventions, and evidence of links between early aggressive behaviour and later violence.4 5 For example, there was some evidence that effects on aggressive behaviour in 4Rs began to emerge after the second intervention year.19 While findings were somewhat contradictory across different outcomes for PATHS, there was some evidence that teachers of intervention students reported less aggression in later years of the intervention.38 Another key feature of Positive Action was the use of a model that linked academic and health education to developmental concerns. That is to say, this intervention focused on improvements in academic engagement and study skills both enhancing, and being enhanced by, student health and well-being; this was a feature of intervention activities and of the underlying theory of change. Moving forward, intervention strategies that combine multiple domains over several years and that use both subject-specific learning alongside linking to developmental concerns may be more effective than classroom-only interventions, single-year interventions or interventions that use literature alone; this should be a target for future research.

This systematic review has strengths and limitations. Identifying relevant studies was challenging often because of poor intervention description. We were unable to undertake meta-analysis or assessment of publication bias, though the preponderance of null results suggests that projects with non-significant findings are being published. Finally, the diversity of outcome measures and of raters precludes a complete and consistent picture of the effectiveness of these interventions via standardised measures. For example, measures that included physical violence and aggression were at times combined with verbal forms of interpersonal violence; while we preferred measures of physical violence and physical aggression, we included outcomes where these behaviours were included as part of a composite. Consistency and clarity in outcome reporting will be especially important as ‘core outcome sets’ become relevant in planning evaluations in public health and social science. Most studies focused on bullying, while evaluations of Positive Action43 45 generally provided the most direct test of violent behaviours specifically.

Future research should seek to understand better the life course aspects of these interventions: that is, how does early school-based intervention impact later-life violent behaviours? From a policy perspective, it is clear that the integration of academic and health education, while possibly an effective intervention, will need to be considered alongside interventions involving other systems to prevent violence. Future evaluations will also contribute by considering the effects of integration in a diversity of ways and mechanisms of action for integration in different types of academic education. For example, contrasts between full and partial integration, which included evaluations did not address, could inform an understanding of how much integration is necessary to support health education messages.

## Acknowledgments

The authors acknowledge Ms Claire Stansfield for her assistance in designing and conducting the searches.

## Footnotes

*   Contributors GJMT undertook study screening and selection, led the meta-analyses and drafted the initial manuscript. TT undertook study screening and selection, extracted data and contributed to drafting the initial manuscript. AF undertook study screening and selection. JT and RC provided methodological and substantive advice. CB undertook study screening and selection, extracted data and contributed to drafting the initial manuscript. All authors revised the manuscript and approved the final manuscript as submitted.

*   Funding This work was funded through National Institute for Health Research Public Health Research Programme grant 14/52/15. The work was undertaken with the support of The Centre for the Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer), a UKCRC Public Health Research Centre of Excellence. Joint funding (MR/KO232331/1) from the British Heart Foundation, Cancer Research UK, Economic and Social Research Council, Medical Research Council, the Welsh Government and the Wellcome Trust, under the auspices of the UK Clinical Research Collaboration, is gratefully acknowledged.

*   Disclaimer The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

*   Competing interests None declared.

*   Patient consent Not required.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data sharing statement All data are publicly available.

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: [https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/).

## References

1.  1. Krug EG , Mercy JA , Dahlberg LL , et al . The world report on violence and health. The Lancet 2002;360:1083–8.[doi:10.1016/S0140-6736(02)11133-0](http://dx.doi.org/10.1016/S0140-6736(02)11133-0) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/s0140-6736(02)11133-0&link_type=DOI) 

2.  2. Scott S , Knapp M , Henderson J , et al . Financial cost of social exclusion: follow up study of antisocial children into adulthood. BMJ 2001;323:191.[doi:10.1136/bmj.323.7306.191](http://dx.doi.org/10.1136/bmj.323.7306.191) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjMvNzMwNi8xOTEiO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vOC85L2UwMjA3OTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.  3. Beinart S , Anderson B , Lee S , et al . Youth at Risk?: A National Survey of Risk Factors, Protective Factors and Problem Behaviour among Young People in England, Scotland and Wales (JRF Findings 432). York: Joseph Rowntree Foundation, 2002.
    
    

4.  4. Bender D , Lösel F . Bullying at school as a predictor of delinquency, violence and other anti-social behaviour in adulthood. Crim Behav Ment Health 2011;21:99–106.[doi:10.1002/cbm.799](http://dx.doi.org/10.1002/cbm.799) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1002/cbm.799&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21370295&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 

5.  5. Olweus D . Bullying at school: What we know and what can we do. Malden, MA: Blackwell, 1993.
    
    

6.  6. Farrington DP , Ttofi MM . School-based programs to reduce bullying and victimization: Campbell Systematic Reviews, 2010.
    
    

7.  7. Hahn R , Fuqua-Whitley D , Wethington H , et al . Effectiveness of universal school-based programs to prevent violent and aggressive behavior: a systematic review. Am J Prev Med 2007;33:S114–S129.[doi:10.1016/j.amepre.2007.04.012](http://dx.doi.org/10.1016/j.amepre.2007.04.012) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.amepre.2007.04.012&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17675013&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000248916600011&link_type=ISI) 

8.  8. Vreeman RC , Carroll AE . A systematic review of school-based interventions to prevent bullying. Arch Pediatr Adolesc Med 2007;161:78–88.[doi:10.1001/archpedi.161.1.78](http://dx.doi.org/10.1001/archpedi.161.1.78) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archpedi.161.1.78&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17199071&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000243273800011&link_type=ISI) 

9.  9. Fletcher A , Bonell C , Sorhaindo A . "We don’t have no drugs education": The myth of universal drugs education in English secondary schools? Int J Drug Policy 2010;21:452–8.[doi:10.1016/j.drugpo.2010.09.009](http://dx.doi.org/10.1016/j.drugpo.2010.09.009) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.drugpo.2010.09.009&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21036025&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000285222500003&link_type=ISI) 

10. 10.NASUWT. English Baccalaureate Survey Summary. Birmingham: NASUWT, 2011.
    
    

11. 11.PSHE Association.  *Comments on the National Curriculum proposals published in* February 2013 *from the PSHE education Strategic Partners’ Group* : PHSEA, 2013.
    
    

12. 12.The Office for Standards in Education Children’s Services and Skills. School inspection handbook: Handbook for inspecting schools in England under section 5 of the Education Act 2005. Manchester: The Government of the United Kingdom, 2016.
    
    

13. 13. Bier MC , Zwarun L , Fehrmann Warren V . Getting universal primary tobacco use prevention into priority area schools: a media literacy approach. Health Promot Pract 2011;12:152S–8.[doi:10.1177/1524839911414887](http://dx.doi.org/10.1177/1524839911414887) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1177/1524839911414887&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22068578&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 

14. 14. Pearson M , Chilton R , Wyatt K , et al . Implementing health promotion programmes in schools: a realist systematic review of research and experience in the United Kingdom. Implement Sci 2015;10:1.[doi:10.1186/s13012-015-0338-6](http://dx.doi.org/10.1186/s13012-015-0338-6) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/s13012-014-0195-8&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=25567289&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 

15. 15. Jones SM , Brown JL , Lawrence Aber J . Two-year impacts of a universal school-based social-emotional and literacy intervention: an experiment in translational developmental research. Child Dev 2011;82:533–54.[doi:10.1111/j.1467-8624.2010.01560.x](http://dx.doi.org/10.1111/j.1467-8624.2010.01560.x) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1111/j.1467-8624.2010.01560.x&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21410922&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000288760000007&link_type=ISI) 

16. 16. Masten AS , Cicchetti D . Developmental cascades. Dev Psychopathol 2010;22:491–5.[doi:10.1017/S0954579410000222](http://dx.doi.org/10.1017/S0954579410000222) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1017/S0954579410000222&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20576173&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 

17. 17. Goleman D . Emotional intelligence. New York: Bantam, 1995.
    
    

18. 18. Jones SM , Brown JL , Aber JL . Three Year Cumulative Impacts of the 4Rs Program on Children’s Social-Emotional, Behavioral, and Academic Outcomes. Society for Research on Educational Effectiveness 2010.
    
    

19. 19. Jones SM , Brown JL , Hoglund WL , et al . A school-randomized clinical trial of an integrated social-emotional learning and literacy intervention: impacts after 1 school year. J Consult Clin Psychol 2010;78:829–42.[doi:10.1037/a0021383](http://dx.doi.org/10.1037/a0021383) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1037/a0021383&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21114343&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 

20. 20.The British Heart Foundation. Money to Burn lesson plan, 2014.
    
    

21. 21. Wright G , Ainsworth P . Plastered evaluation: part of It’s Not OK! Violence Prevention Education Programme. Ariel Trust: Liverpool, 2008.
    
    

22. 22. Flay BR , Graumlich S , Segawa E , et al . Effects of 2 prevention programs on high-risk behaviors among African American youth: a randomized trial. Arch Pediatr Adolesc Med 2004;158:377–84.[doi:10.1001/archpedi.158.4.377](http://dx.doi.org/10.1001/archpedi.158.4.377) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archpedi.158.4.377&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15066879&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000220684700016&link_type=ISI) 

23. 23. Kupersmidt JB , Scull TM , Benson JW . Improving media message interpretation processing skills to promote healthy decision making about substance use: the effects of the middle school media ready curriculum. J Health Commun 2012;17:546–63.[doi:10.1080/10810730.2011.635769](http://dx.doi.org/10.1080/10810730.2011.635769) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1080/10810730.2011.635769&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=22339322&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000304481200005&link_type=ISI) 

24. 24. Patton GC , Bond L , Carlin JB , et al . Promoting social inclusion in schools: a group-randomized trial of effects on student health risk behavior and well-being. Am J Public Health 2006;96:1582–7.[doi:10.2105/AJPH.2004.047399](http://dx.doi.org/10.2105/AJPH.2004.047399) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.2105/AJPH.2004.047399&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16873760&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000240179100018&link_type=ISI) 

25. 25. Peersman G , Oliver S , Oakley A . EPPI-Center review guidelines: data collection for the EPIC database. London: EPPI-Centre Social Science Research Unit, 1997.
    
    

26. 26. Higgins JPT , Green S . Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration: Oxford, 2011. [updated March 2011].
    
    

27. 27. Sutcliffe K , Thomas J , Stokes G , et al . Intervention Component Analysis (ICA): a pragmatic approach for identifying the critical features of complex interventions. Syst Rev 2015;4:140.[doi:10.1186/s13643-015-0126-z](http://dx.doi.org/10.1186/s13643-015-0126-z) 
    
    

28. 28. Bond L , Patton G , Glover S , et al . The Gatehouse Project: can a multilevel school intervention affect emotional wellbeing and health risk behaviours? J Epidemiol Community Health 2004;58:997–1003.[doi:10.1136/jech.2003.009449](http://dx.doi.org/10.1136/jech.2003.009449) 
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiamVjaCI7czo1OiJyZXNpZCI7czo5OiI1OC8xMi85OTciO3M6NDoiYXRvbSI7czoyNToiL2Jtam9wZW4vOC85L2UwMjA3OTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

29. 29. Aber JL , Tubbs C , Torrente C , et al . Promoting children’s learning and development in conflict-affected countries: Testing change process in the Democratic Republic of the Congo. Dev Psychopathol 2017;29:53–67.[doi:10.1017/S0954579416001139](http://dx.doi.org/10.1017/S0954579416001139) 
    
    

30. 30. Torrente C , Johnston B , Starkey L , et al . Improving the quality of school interactions and student well-being: Impacts of one year of a school-based program in the democratic republic of the congo. Journal on Education in Emergencies 2015;1:48–91.
    
    

31. 31. Espelage DL , Low S , Van Ryzin MJ , et al . Clinical trial of second step middle school program: Impact on bullying, cyberbullying, homophobic teasing, and sexual harassment perpetration. School Psych Rev 2015;44:464–79.[doi:10.17105/spr-15-0052.1](http://dx.doi.org/10.17105/spr-15-0052.1) 
    
    

32. 32. Espelage DL , Low S , Polanin JR , et al . Clinical trial of Second Step© middle-school program: Impact on aggression & victimization. J Appl Dev Psychol 2015;37:52–63.[doi:10.1016/j.appdev.2014.11.007](http://dx.doi.org/10.1016/j.appdev.2014.11.007) 
    
    

33. 33. Espelage DL , Low S , Polanin JR , et al . The impact of a middle school program to reduce aggression, victimization, and sexual violence. J Adolesc Health 2013;53:180–6.[doi:10.1016/j.jadohealth.2013.02.021](http://dx.doi.org/10.1016/j.jadohealth.2013.02.021) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jadohealth.2013.02.021&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23643338&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000322065100005&link_type=ISI) 

34. 34. Couch L . The Bullying Literature Project: An Evaluation of a Class-Wide Bullying Intervention Program. Education. California: University of California Riverside, 2015.
    
    

35. 35. Jenson JM , Dieterich WA . Effects of a skills-based prevention program on bullying and bully victimization among elementary school children. Prev Sci 2007;8:285–96.[doi:10.1007/s11121-007-0076-3](http://dx.doi.org/10.1007/s11121-007-0076-3) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=17968657&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 

36. 36. Li KK , Washburn I , DuBois DL , et al . Effects of the Positive Action programme on problem behaviours in elementary school students: a matched-pair randomised control trial in Chicago. Psychol Health 2011;26:187–204.[doi:10.1080/08870446.2011.531574](http://dx.doi.org/10.1080/08870446.2011.531574) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=21318929&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 

37. 37. Frey KS , Hirschstein MK , Snell JL , et al . Reducing playground bullying and supporting beliefs: an experimental trial of the steps to respect program. Dev Psychol 2005;41:479–90.[doi:10.1037/0012-1649.41.3.479](http://dx.doi.org/10.1037/0012-1649.41.3.479) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1037/0012-1649.41.3.479&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15910156&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000229331800003&link_type=ISI) 

38. 38. Crean HF , Johnson DB . Promoting Alternative Thinking Strategies (PATHS) and elementary school aged children’s aggression: results from a cluster randomized trial. Am J Community Psychol 2013;52:56–72.[doi:10.1007/s10464-013-9576-4](http://dx.doi.org/10.1007/s10464-013-9576-4) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1007/s10464-013-9576-4&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23625456&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000322647000005&link_type=ISI) 

39. 39. Reid JB , Eddy JM , Fetrow RA , et al . Description and immediate impacts of a preventive intervention for conduct problems. Am J Community Psychol 1999;27:483–518.[doi:10.1023/A:1022181111368](http://dx.doi.org/10.1023/A:1022181111368) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1023/A:1022181111368&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=10573832&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000083644000003&link_type=ISI) 

40. 40. Langford R , Bonell CP , Jones HE , et al . The WHO Health Promoting School framework for improving the health and well-being of students and their academic achievement. Cochrane Database Syst Rev 2014;4:CD008958.[doi:10.1002/14651858.CD008958.pub2](http://dx.doi.org/10.1002/14651858.CD008958.pub2) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=24737131&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 

41. 41. Wang C , Goldberg TS . Using children’s literature to decrease moral disengagement and victimization among elementary school students. Psychol Sch 2017;54:918–31.[doi:10.1002/pits.22042](http://dx.doi.org/10.1002/pits.22042) 
    
    

42. 42. Brown EC , Low S , Smith BH , et al . Outcomes from a school-randomized controlled trial of steps to respect: A bullying prevention program. School Psychology Review 2011;40:423.
    
    

43. 43. Beets MW , Flay BR , Vuchinich S , et al . Use of a social and character development program to prevent substance use, violent behaviors, and sexual activity among elementary-school students in Hawaii. Am J Public Health 2009;99:1438–45.[doi:10.2105/AJPH.2008.142919](http://dx.doi.org/10.2105/AJPH.2008.142919) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.2105/AJPH.2008.142919&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19542037&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000268874100022&link_type=ISI) 

44. 44. Jenson JM , Brisson D , Bender KA , et al . Effects of the Youth Matters Prevention Program on Patterns of Bullying and Victimization in Elementary and Middle School. Soc Work Res 2013;37:361–72.[doi:10.1093/swr/svt030](http://dx.doi.org/10.1093/swr/svt030) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1093/swr/svt030&link_type=DOI) 

45. 45. Lewis KM , Schure MB , Bavarian N , et al . Problem behavior and urban, low-income youth: a randomized controlled trial of positive action in Chicago. Am J Prev Med 2013;44:622–30.[doi:10.1016/j.amepre.2013.01.030](http://dx.doi.org/10.1016/j.amepre.2013.01.030) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.amepre.2013.01.030&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23683980&link_type=MED&atom=%2Fbmjopen%2F8%2F9%2Fe020793.atom) 

46. 46. Jenson JM , Dieterich WA , Brisson D , et al . Preventing childhood bullying: Findings and lessons from the denver public schools trial. Res Soc Work Pract 2010;20:509–17.[doi:10.1177/1049731509359186](http://dx.doi.org/10.1177/1049731509359186) 
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1177/1049731509359186&link_type=DOI) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000280611400008&link_type=ISI)