Accountability, Race, and Class

by Joshua Otlin, Principal – Milford High School    @jcotlin

Median RankI am a first-year principal at Milford High School, a Level 3 school ranked at the 19th percentile.  I am expected to help the school receive a Level 2 rating, so I have studied the system of ratings and rankings to inform my action plan.  I want to share some of what I have learned because I am troubled at what I found: for the vast majority of schools, demography is destiny, as the above chart illustrates.  In other words, level ratings and percentile ranks are almost exclusively a measure of the concentration of disadvantage in a school, not school quality.

My conclusions are drawn from the most recent accountability data on the 253 high schools with both a level rating and percentile rank.  The tables and charts here include data from 208 “general” high schools and exclude 45 schools with selection-bias:  charters, pilots, exam schools, and vocational schools (click here for the full data set, including all 253 high schools with a rating and a rank).[i]  Including those schools adds more outliers, but does not alter the findings:

Schools with few Economically Disadvantaged (ED) students are immune from negative ratings

Every school with less than 25% ED students received a Level 1 or 2 rating, even if the school was outranked by Level 3 schools with more than 25% ED students.  Conversely, there are schools with 25%+ ED students that match or outrank schools with less than 25% ED students but receive a lower rating.  The following table shows the 16 schools with a rank between 21-30, the range where Level 2 and 3 schools overlap.

Rank School Name Rating ED %
30 Waltham Senior High Level 3 35.6
29 Woburn High Level 2 23.6
28 Westfield High Level 3 26.6
28 Brockton High Level 2 49.4
27 Rockland Senior High Level 1 24
27 Methuen High Level 3 31.9
27 Wareham Senior High Level 2 41.9
26 Plymouth South High Level 2 16.9
25 Gloucester High Level 2 34.5
24 Attleboro High Level 2 27.7
24 Worcester South High Level 3 61.1
23 Leominster High School Level 2 29.5
23 Everett High Level 2 43.9
22 Plymouth North High Level 2 22.5
21 E Brookfield (David Prouty) Level 2 33.4
21 Salem High Level 2 46.1

A school’s concentration of poverty is strongly correlated with the school’s rank

This scatterplot of the 208 general high schools shows the clear relationship between poverty and rank (click here for an interactive version including all 253 high schools in the data set):

Bubble Graph

The association is very strong, but there are also numerous schools serving more than 25% ED students that outperform their demographic peers and received favorable ratings.  Of course, this is also true for schools with less than 25% ED students, except all of those schools received favorable ratings.  The schools serving 25%+ ED student and receiving favorable ratings surely have something to teach us about preparing disadvantaged students for grade 10 MCAS.  They do not, however, change the fact that the odds of earning favorable ratings diminish dramatically as the concentration of disadvantage increases.

Racial achievement gaps are an illusion

An initial look at race suggests a strong association with ratings:

Racial Composition # of Schools % of Schools at Levels 3-4
75%+ White 120 4%
25%+ African-American 13 77%
25%+ Hispanic 39 87%
50%+ Non-White 52 92%

 

A closer examination, however, reveals that the initial analysis is misleading.  When we look at poverty, too, we see that race is not so tightly linked to rating.

Race & Poverty Bars

We know that race is strongly associated with income in Massachusetts, so this finding is unsurprising.  We do see, however, some important differences when we take a more nuanced look at race:

Association with Rank (P < .0001)
Factor R-Square
% African-American 0.19
% White 0.36
% Hispanic 0.46
% ED 0.75

There is a relatively strong association between the concentration of Hispanic students in a school and that school’s rank, although not nearly as strong as the association between rank and ED students.  Hispanics students, on average, are the most disadvantaged in Massachusetts: they are most likely to come from extremely poor households and a significant number are English-learners (see here for more on race and income in Massachusetts).

Discussion

When we look at the intersections between race and poverty and accountability, we see that ratings and rankings simply reflect patterns of privilege and power in our society.  As long as we maintain the status quo, demography will dictate results for the vast majority of schools.  For a school like Milford, one of 39 general high schools with at least 25% Hispanic and 25% ED students, this spells trouble.  87% of these schools received a Level 3 or 4 rating and the median rank was 11 (when including schools with selection-bias and at least 25% Hispanic students, the median rank is still only 15).  Knowing this, I will still do my best to help our school receive a Level 2 rating, but I do so with the understanding that my odds are quite poor.

While I remain opposed to the use of standardized tests for rating and ranking schools, I accept that there is little prospect of change and hope that system might at least be reformed to be marginally less bad.  To do so, I suggest that we employ the “concentration of disadvantage” in the same manner that we use “degree of difficulty” in a number of sports.  Schools with high concentrations of disadvantage work at a high degree of difficulty with regards to test-based accountability.  The system, however, rests on the assumption that there is no such difference between schools.  This is unrealistic and irresponsible.  Developing a model that accounts for these difference would be no more complicated than analogous scoring systems in athletics.

Conclusion

Ratings and rankings do not tell us anything meaningful about school quality.  They largely reflect the concentration of disadvantage in a school, and to a lesser extent for schools with 25%+ ED students, they tell us about the efficacy of the school’s MCAS preparation program.  If I am wrong, however, and the system accurately measures school quality, then the data clearly demonstrates the failure of 20+ years spent trying to reduce achievement gaps through test-based accountability.  Either way, it’s past time for something new.

About The Author

JoshJoshua Otlin is a first-year principal at Milford High School, his alma mater.  Prior to coming to Milford, he most recently worked as an Assistant Principal at Hudson High School, where he also served a data specialist for the district.  Joshua has worked in Massachusetts’s public high schools for 18 years, teaching Social Studies before becoming an administrator.

 

[i] Comparing “general” high schools to schools with selection-bias is inappropriate and adds to the misinformation surrounding accountability measures, but I have conducted multiple rounds of analysis, both with and without schools with selection-bias, and my key findings remain unchanged.  I encourage you to download the data set and see for yourself.

Write for the MSAA Blog

Are you interested in sharing your ideas, insights and questions with colleagues? You don’t need to be an experienced blogger.  Consider reworking a piece you wrote for staff or families. For more information or to sign-up to write a post, contact rrogers@msaa.net.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s