What Chicago’s Schools Want to Evaluate In the District’s Own Words
Dylan Matthews lays out the rift between Chicago’s teachers and the city’s management in some numerical detail. Bottom line, teachers are being asked to do more with less. Less pay, more accountability, but still no real authority. As the education reform movement has made clear, teachers are to be the most responsible for student achievement, despite having more and more of the education agenda forced on them from the top down.
Here are the vague particulars of Chicago’s suggested teacher evaluation regime. Like all good government ideas, it comes with a positively platitudinous acronym: REACH (or Recognizing Educators Advancing CHicago’s Students (Yes. The schools in Chicago are so bad that even the city’s elites don’t know how acronyms work[/efn_note].
The document makes sure to stress just how “research-based” their methods for evaluation are. I’m glad. Voodoo, astrology, and other methods would probably not be as accurate.
It cites the “The Gates’ Foundations MET study,” noting how it helped establish “the strong correlations between classroom observation ratings using a Danielson-based framework and student learning outcomes.” Danielson herself has been cautious about using her framework in “high-stakes” type situations, saying in 2011 that,
“What I hope people guard against is, so long as practice is above a certain level, then it shouldn’t be high-stakes. If you aren’t going to fire the person, then what’s the point? Some people who are driving this policy have a “get rid of the bad apples” mentality, but I’m [not sure there are sufficient replacement teachers out there]. If we assume that most of these teachers right now are still going to be on staff in five years time, then the challenge is how do we get better? And that entails very different procedures and a different culture than it does if your goal is to smoke out the bad apples.”
Of course no one is talking about bad apples here, right? We’re all friends. It’s about the kids after all.
And that Gate’s Foundation MET study? Well it was a runner-up in the “Mirror Image” category in the University of Colorado’s Annual Bunkum Awards. That’s because the study “reached a conclusion that was the exact opposite of what the evidence suggested.” A lengthy critique can be read here.
Next, the press release says,
“Our own experience tells us that teaching is complex and that any one measure of a teacher won’t necessarily tell us the whole story. The research strongly supports the importance of including multiple measures of teacher performance in any teacher evaluation system.”
Well I’m relieved. For a moment I thought they were going to be evaluating teachers according to a single metric. Bring on the nuance,
“Each study uses one measure of student learning in its assessment of teachers, but CPS will have two measures of student learning—standardized test and performance task.”
How did this new evaluative scheme come about?
“To create this enhanced system, the CTU and CPS established two committees that have been in negotiations since November 2011. Between November 2011 and April 2012, the two committees held over 35 meetings and met for a total of over 90 hours to discuss all elements of the new system.”
In other words, about two weeks. Where I work we take longer than that to design a gearbox in which that’s based on over a hundred years of data and experience, as well as exact sciences like, for instance, physics.
The goal or “end state” Chicago schools are aiming for is one in which evaluations are comprised of “50 percent teacher practice, 40 percent student growth and 10 percent student feedback.”
The release outlines a lot more empty rhetoric before moving on to something substantive (which doesn’t start till page 5 no less),
“In grades 3-8, CPS will use the Northwest Evaluation Association Measures of Academic Progress (NWEA MAP) assessment. In grades 9-11, CPS will use the EPAS series of assessments: EXPLORE in Grade 9, PLAN in Grade 10, and ACT in Grade 11.”
Never mind that explore, plan, act happens to be how I organize a night of Chinese takeout with my roommates. This is serious stuff. Our kids deserve the best (which in this case seems to be an acronym for every grade)*.
By this point you might be wondering what exactly the difference is between the value-added and the expected gains approaches. The latter “controls for incoming test score, but not the other demographic factors ,” but “allows growth targets to be set based on beginning of year results” so that growth can be “measured as the percentage of students meeting or surpassing anticipated growth at the end of year administration.”
Value-added on the other hand, “isolates a teacher’s impact on student achievement by separating out the effect of variables outside of the teacher’s control.” What are these variables?
Well, despite the fact that “measuring that growth accurately and objectively is difficult for a variety of reasons,” the administrators believe that growth based measures like the value-added approach are best because they take “students’ prior achievement as well as other student characteristics” into account.
What these characteristics are still remains a mystery. Don’t worry though, because “With reliability between .30 and .50, VAM is directly in line with the best measures in education.” .30-.50? Is that the correlation value? Please tell me it’s something else.
And then the kicker,
“CPS is taking into account the best research and using value-added in the most appropriate manner possible; blending student data with classroom observation. Such an approach will yield an assessment system that is akin to using a meter stick to measure height at the beginning of the year and then again at the end of the year to understand how much a child grew.”
Trust us, we are professionals, we know what we’re doing. Not only that, say the schools, but remember when we told you about how measuring a teacher’s impact on a child’s education is complicated and difficult? Well actually it’s kinda more like when you go to the doctor’s office and get your height, weight, and temperature taken.
The value-added data is so important and helpful that the schools firmly believe such information should be kept a secret.
Then there is the student survey component of evaluation,
“Research confirms what many of us would instinctively say – that students are important and accurate evaluators of teacher effectiveness.”
Indeed. That is all I will say on this matter. Mostly because CPS says nothing more on the subject, like what kinds of questions will be asked, whether it will be closer to the student evaluations most professors receive at the end of a semester, or closer to the customer satisfaction ones people either do or do not complete anytime they’ve been forced to call a company’s help line or return retail merchandise.
I plan on having several follow-up posts in the days and weeks to come. The presidential election is a joke, but the fight between “reform” and public unions will only be heating even as the weather continues to get cooler.
What will be more interesting to watch is how the two sides’ rhetoric develops and who is reported on more favorably. I’m not betting on the unions, especially with misleading arguments being made throughout the blogosphere that Chicago’s teachers are being made a “pretty plum offer” and that test scores and student achievement will only have “some role in evaluating teachers.”
The truth is that a 2-3% annual raise is really a loss when you factor in inflation and the growth scales that others in the private sector (like myself) enjoy. 3% of 47K a year is nothing, especially when you’re being asked to educated one of the most disadvantaged student populations in the country.