Exemplars Performance Materials for Mathematics Meet ESSA’s Tier 3 Standards
Brief ESSA History
Passed in 2015, the goal of the Every Student Succeeds Act (ESSA), is to ensure that all students have access to an equitable education that fully prepares them for success in college and careers.1
ESSA incorporates the concept of “evidence-based” into legislation to ensure that the approaches and programs being implemented effectively improve educational outcomes and close achievement gaps. In order for a school or district to spend federal funds on educational products or services, ESSA requires that the product or service meets its standards.
We are pleased to announce that ReContext Data Solutions, LLC, an independent, outside research organization, has determined that Exemplars performance material in mathematics is “evidence-based” and meets ESSA’s Tier 3 standards.
To demonstrate a basis for Tier 3, ReContext Data Solutions conducted a study, in alignment with ESSA requirements, to determine the associations between the use of Exemplars products and positive outcomes for students in mathematics.
Tier 3 Study Overview
The study set out to answer the following questions:
- Do schools that use Exemplars products experience better student outcomes in mathematics than those that do not?
- Do schools that use Exemplars products for longer periods of time experience better outcomes than those that use them for shorter periods of time or those that do not use them?
- Are any observed gains due to confounding effect from other influences, such as student demographics, or are they more greatly associated with product use?
The study was confined to public schools within the New York City school system. All public schools in New York City were considered for inclusion in the analysis. Further, however, it was confined to schools that served at least one grade of students between grades 3 and 8 for all of the years 2016 through 2018. Data including name, location, and achievement for schools was downloaded from the New York State Education Data web site (data.nysed.gov). Data on Exemplars customers was provided by the company and includes the names of schools and the years that they used the products.
1,246 New York City public schools met the research requirements, of which 131 used Exemplars products for at least one of the studied school years. The demographic difference between the schools that used Exemplars and those that did not were minimal as illustrated in Figure 1 below.
| Exemplars | Non-Exemplars | |
|---|---|---|
| Schools | 131 | 1115 |
| Students | 39,094 | 376,442 |
| Students Per School | 298 | 338 |
| EDS | 56% | 60% |
| LEP | 13% | 11% |
| IEP | 20% | 18% |
Figure 2 shows the achievement of New York City schools on the NYS Mathematics Exam, based on the number of years of Exemplars product implementation.
As evident in Figure 2, schools demonstrated higher gains the longer they used Exemplars math performance materials, with a plateau in Year 4. This can be further seen in Figure 3.
The small difference between Year 1 schools and Non-Exemplars schools is not significant, based on α testing, but the clear differences in years 2 and 3 are signs of gains in those years.
Demographic Norming
It is a well-known phenomenon that schools serving fewer students of low socioeconomic status tend to score higher on test-based aggregated measures of performance. This study evaluated the potential impact of demographic, grade span, and test variability on school performance scores. By adjusting for grade level, student demographics (EDS, IEP and ELL), and test variability, the study affirmed that the gains that schools, which use Exemplars products, cannot be explained by differences in demography or grade-level representation.
Schools that use Exemplars products for longer periods of time tend to outperform demographically similar institutions.
Results of Study
Conclusions from this study are based on the initial research questions:
- Question 1: Do schools that use Exemplars products experience better student outcomes in mathematics than those that do not?
Findings: As observed on both measures of proficiency and normed-scaled score, schools that use Exemplars outperform schools that are not using the products on NY State Mathematics exams.
- Question 2: Do schools that use Exemplars products for longer periods of time experience better outcomes than those that use them for shorter periods of time or those that do not use them?
Findings: The observed phenomenon gets stronger with longer use of Exemplars within a given school. We observed a plateau between years 3 and 4, but further study is warranted to determine if the benefit continues to grow beyond year 4.
- Question 3: Are any observed gains due to the confounding effect from other influences, such as student demographics, or are they more greatly associated with product use
Findings: By measuring both the demographic similarity between the control and experimental groups and by using a normed measure of performance, the study demonstrates that the benefit experienced by Exemplars schools is not due to student demographics or similar confounding variables.
Exemplars Meets ESSA Tier 3 Standards
On average, schools using Exemplars show gains on the NYS Mathematics Exams, with the steepest gains coming in years 2 and 3 of implementation. It is the opinion of ReContext Data Solutions that Exemplars Math Material has satisfied the standards of Tier 3 as set forth in the Every Student Succeeds Act.