Are individual/small group interventions in math more effective than class/school level interventions? A meta-analysis of What Works Clearinghouse reviews

ID: 

2126

Session: 

Poster session 2 Thursday: Evidence synthesis - methods / improving conduct and reporting

Date: 

Thursday 14 September 2017 - 12:30 to 14:00

Location: 

All authors in correct order:

Chan T1, Streke A2
1 AIR, USA
2 Mathematica Policy Research, USA
Presenting author and contact person

Presenting author:

Tsze Chan

Contact person:

Abstract text
Background: K-12 interventions are often delivered to targeted students in groups of various sizes – individuals, small groups, the entire class or school. A systematic review of how does mode of delivery affect the effectiveness of interventions in Math has not been conducted (Bloom, 1984; for Reading, Slavin et al. 2009). What Works Clearinghouse (WWC) has recently made available to the public coded database of reviewed evaluation studies. One of the study properties included is delivery of intervention at individual, small group, classroom and school levels.

Objectives:
1. Address the following research hypothesis: In the content area of Math, are interventions that target individual/small groups more effective than interventions that serve class or larger groups?
2. Discuss the advantages and challenges of using the WWC database for works in meta-analysis.


Methods: To address the research question, we conduct a meta-analysis on the recently released WWC database on reviewed studies in the area of math, and compare the effect sizes of K-12 interventions serving individual/small groups with interventions that serve at the class or school level. The analysis data set consists of 226 effect sizes and 67 reviewed studies. The comparison is made in a random-effect model by meta-regression technique. Our study uses the robust variance estimation procedure to adjust for the within study dependencies among effect sizes (Hedges et al. 2010; Tanner-Smith et al. 2014). Challenges encountered when using the WWC database for this analysis will also be described.

Results: Our results show, that after controlling for sample size and other variables, individual/small group interventions are more effective than interventions targeting classrooms and schools.

Conclusions: Evidence suggests that individual/small group interventions are better in producing positive math outcomes than interventions that target larger groups. We will also outline the pros and cons, tips and recommendations when using the WWC publicly available data for meta-analysis.