The Economic Policy Institute agrees with something I’ve been saying for a while: “American public schools generally do a poor job of systematically developing and evaluating teachers.”
So what does it recommend we should do? In a press release, it says, “While there are good reasons for concern about the current system of teacher evaluation, there are also good reasons to be concerned about claims that measuring teachers’ effectiveness largely by student test scores will lead to improved student achievement.”
So what’s the place for value-added assessments? It says making them half of an evaluation system is “unwise.” In a blog post, the institute’s Andrea Orr says, “the accuracy of these analyses of student test scores is highly problematic.”
A 29-page briefing paper (PDF) published by the institute says that change in test scores “should be used only as a modest part of a broader set of evidence about teacher practice.” It mentions several objections; I list them in a rough order of decreasing important persuasiveness, in my opinion:
- When value-added models have been used, their performance over time suggests that they’re not quite ready for prime time: “One study found that across five large urban districts, among teachers who were ranked in the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40%.” Now that’s a problem. While an individual teacher’s effectiveness can change over time, when such a large number of teachers change over just one year, the methodological validity of the measurement is called into question.
- There are many non-school factors at work: “Student test score gains are also influenced by family resources, student health, family mobility, and the influence of neighborhood peers and of classmates who may be relatively more advantaged or disadvantaged.”
- Student test score gains are also influenced by family resources, student health, family mobility, and the influence of neighborhood peers and of classmates who may be relatively more advantaged or disadvantaged.
- A variety of other factors inside the school can affect student test scores. They include: “curriculum materials, specialist or tutoring supports, class size, and other factors.” (OK, then, how about giving bonuses to the entire school staff if students excel? That recognizes the role of employees beyond the classroom teacher.)
- A student’s performance in subject X may be effected by his performance in subject Y; as a result, the measured effectiveness of one teacher may be tied up by the measured effectiveness of another.
- When schools use team teaching, block scheduling, or some other techniques, it’s hard to single out the contribution of a specific student.
The report admits that “Used with caution, value-added modeling can add useful information.”
Though I’m fairly sure the authors of the report did not mean to endorse school choice, their conclusion offers some support for that option: “Yet there are many alternatives that should be the subject of experiments. The Department of Education should actively encourage states to experiment with a range of approaches that differ in the ways in which they evaluate teacher practice and examine teachers’ contributions to student learning. These experiments should all be fully evaluated.”
A wide range of approaches? Well, that’s what we use in automobiles, where consumers can choose from the reviews offered by Car & Driver, Edmunds, Kellys, the Institute for Highway Safety and so forth. For service companies, you can look at reviews provided by Angie’s List, Yelp. Consumer Reports, and others. Yet in schooling, we’re trying to find the one reliable and valid approach to evaluating teachers.
Why? Because unlike the cases of automobiles or carpet cleaners or doctors, when it comes to schooling, we expect everyone in a given geographic area to purchase from the same provider–the local school district. Naturally, there’s going to be an incredible amount of controversy over what the appropriate metric of quality is or should be.
I agree with the authors who say, “School districts should be given freedom to experiment, and professional organizations should assume greater responsibility for developing standards of evaluation that districts can use.” But that freedom will be most effective in a free market for schooling, with significant choice among schools, and competition for student dollars. That’s a far cry from what we have now.