Test scores, learning rates tell different student achievement stories in Michigan’s poorest schools

A recent study shows a concept that we have seen for a long time — children’s scores are much higher in richer districts in Michigan. But the study also shows that test scores don’t paint the whole picture for student achievement when comparing poorer and more affluent schools. 

The Educational Opportunity Project, a study done by researchers at Stanford University, shows the tired tale of poor students scoring lower on standardized tests is still alive in Michigan’s public schools. However, the results also show that some of these poorer districts are improving at rates equal to or higher than more affluent districts. 

The data raise the question of how we compare the effectiveness of schools for districts with less resources.

For years, educators and lawmakers have debated the test scores vs. learning rate conundrum. This fueled much of the change from No Child Left Behind to the Every Student Succeeds Act in 2015 and has redirected the way states evaluate the successes of schools. 

No Child Left Behind, which updated the Elementary and Secondary Education Act in 2002 under former President George W. Bush, focused on student achievement with a priority on state reading and math test scores to evaluate the effectiveness of schools. 

The Every Student Succeeds Act requires school evaluation measures to be factored by reading and math test scores, as well as English-language proficiency test scores and high school graduation rates. It also lets states decide school quality factors, which in Michigan include chronic absenteeism, postsecondary enrollment and advanced coursework. 

The Educational Opportunity Project, which looks at standardized test score data for students in third grade through eighth grade from 2008 to 2016, provides data on three measures of educational opportunity:

• Average test scores, which reflect educational opportunities children have from birth through middle school.
• Learning rates, which measure how much students’ scores improve each year while they are in school. 
• Trends in how much average test scores change each year which indicate whether educational opportunities are improving or declining in a community. 

https://www.michiganadvance.com/2019/09/17/what-m-step-scores-showed-us-about-michigans-education-achievement-gap/

Sean Reardon, a professor at Stanford University and a lead researcher for the Educational Opportunity Project, says that evaluating the effectiveness of a school by test schools is insufficient, because this measure is affected by too many variables outside of the school’s control. 

“Average student test scores are influenced by opportunities to learn at home, in neighborhoods, in child-care, preschool, and after-school programs, from peers and friends, and in schools. Because of all these influences, average test scores are not a good way to assess how much children learn in schools,” Reardon wrote. “But they are a good way to assess the average set of educational opportunities available to children in a community. Where average scores are low, students have fewer opportunities to learn.”

Instead, Reardon suggests schools be evaluated using learning rates.

“To understand the distinction, think of a hospital: We wouldn’t assess a hospital based on the health of its patients; rather, we’d judge it on how much the health of patients improves as a result of their time in the hospital. Similarly, we shouldn’t evaluate a school based on the average scores of its students, but rather by how much their scores improve while in school,” he included in his research.

The trend in average test scores from one year to the next shows how school quality, family and community resources, home environments and early childhood experiences can improve or worsen a student’s opportunity to learn.  

https://www.michiganadvance.com/blog/54-of-michigan-3rd-graders-not-reading-proficiently/

“It’s critical to measure, on the output side, both learning and achievement. Very often the focus is on the achievement only. And we think that there’s importance to having a real balance,” said Brian Gutman, director of external relations at Education Trust-Midwest. 

“Often we speak about these things in silos, and even our systems that exist in Michigan don’t often work well together. And so, what the ultimate hope would be is that we move toward systems that use the use more effective data models, so that we’re able to both measure learning, as well as learning toward achievement.”

Tale of 2 Michigan school districts

In Michigan, the study shows an unexpected story of the quality of public education for two districts on both ends of the socioeconomic scale. The data show that on average, learning rates are modestly higher in more affluent districts than in less affluent ones, but the data reveals some interesting exceptions. 

The socioeconomic status of Detroit Public Schools (DPS) is 3.51 points below the national average, which is described as ‘very far below the national average.’ Detroit Public Schools was since replaced by Detroit Public Schools Community District (DPSCD) in 2016 after the data for this study were collected. 

Less than 30 miles away is West Bloomfield School District, which is 1.35 points about the national average for socioeconomic status. 

https://www.michiganadvance.com/2019/10/30/national-education-test-results-show-small-improvement-statewide-in-detroit/

The Stanford study also finds that achievement gaps in high poverty school districts perpetuates inequality for minority students. So it is important to note that data shows 97.3% of students at DPSCD are minority students, compared to 46.1% at West Bloomfield Public Schools.  

During the years studied, standardized test scores in Detroit were bleak, where students were scoring 2.49 grade levels below the national average. Test scores are still a problem today for Detroit students, according to  national test results released in October that showed that Detroit public schools ranked last among 27 urban areas across the country. 

In West Bloomfield, students scored .91 grade levels above the national average during the years studied. 

From a quick glance, the success of these two schools looks clear. But researchers are urging education leaders to look beyond the test scores and consider growth as a stronger indicator of student achievement. 

While students in Detroit public schools are still learning 6% less each grade than the national average, students in West Bloomfield are learning 13% less each grade. 

This example shows that students in historically poor and underperforming school districts start off testing far below average, but that the schools appear to be progressing and catching up to the national average. It also shows that basing the quality of a school based on community wealth or average test scores can be misleading.

Where DPSCD goes from here 

Nikolai Vitti, the Detroit Public Schools Community District superintendent who joined the district in 2017, says that a school’s success has as much to do with accountability and resources as it does students’ performance on standardized tests.

“Most state accountability systems aren’t structured in the right way. Because it’s creating a one size fits all, win-or-lose scenario based on usually at-or-above grade level performance,” Vitti says. “And I actually believe if we’re really true about making education individualistic, based on the individual needs of the child, then I think the accountability system has to mirror that and have the right balance based on where that individual child or groups of children are at.”

Vitti says that when you compare the “starting point for students” in poorer districts and more affluent districts, it is really showing what the students were exposed to at home and is linked to cultural experiences. 

“Nothing related to that data is about intelligence or potential,” Vitti said. “It’s really a reflection of what children were exposed to while they were in the womb and before they started school.”

Since joining the district, Vitti has been tough on holding educators and other administrators accountable for student growth through new curriculums, training and new standards, but he understands that evaluating students by standardized test scores is a reality for our current education system.

“We had many conversations about performance in DPS and DPSCD, and standardized testing has been used as a weapon against our schools and our children over time, justifying the closure of schools and governance structures that don’t empower local citizens. There’s a deep sense of distrust and skepticism about the role of standardized tests, and I think that’s justified,” Vitti said. 

“At the same time, we’re trying to shift the conversation to say standardized testing is a reality that’s not going away … and our children can do well on tests in general, if they’re equipped with the right background information and skills. And that’s part of our role in the K-12 school system if we’re about making children ready for college and the world of work.”

But it’s also a common understanding that without state and federal assistance for public schools, achievement gaps for students at high-poverty, high-minority-population districts will continue.

“With an accountability structure, you then use that data to define the resources that are necessary. And that’s when you have to start having a different conversation in the state and I think this country about how we allocate resources, where those school districts that need more receive more and those districts that have more receive less,” Vitti said. 

“We haven’t really been able to do that in this country, and especially in Michigan. So what we end up doing is we perpetuate inequality defined through socio-economic status, and often based on the history of racial injustice in this country.”