re: the la times sensationalist expose' series on value-added modeling of LAUSD teacher "effectiveness"
this is a report by eva baker, linda darling-hammond, and many others. necessary context for those newspaper articles.
http://epi.3cdn.net/b9667271ee6c154195_t9m6iij8k.pdf
"Nonetheless, there is broad agreement among statisticians, psychometricians, and economists that student test scores
alone are not sufficiently reliable and valid indicators of teacher effectiveness to be used in high-stakes personnel decisions,
even when the most sophisticated statistical applications such as value-added modeling are employed.
-
"For a variety of reasons, analyses of VAM results have led researchers to doubt whether the methodology can accurately
identify more and less effective teachers. VAM estimates have proven to be unstable across statistical models, years, and
classes that teachers teach. One study found that across five large urban districts, among teachers who were ranked in
the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third
moved all the way down to the bottom 40%. Another found that teachers’ effectiveness ratings in one year could only
predict from 4% to 16% of the variation in such ratings in the following year. Thus, a teacher who appears to be very
ineffective in one year might have a dramatically different result the following year. The same dramatic fluctuations were
found for teachers ranked at the bottom in the first year of analysis. This runs counter to most people’s notions that the true
quality of a teacher is likely to change very little over time and raises questions about whether what is measured is largely
a “teacher effect” or the effect of a wide variety of other factors...
-
"Evaluation by competent supervisors and peers, employing such approaches, should form the foundation of
teacher evaluation systems, with a supplemental role played by multiple measures of student learning gains that,
where appropriate, could include test scores. Some districts have found ways to identify, improve, and as necessary,
dismiss teachers using strategies like peer assistance and evaluation that offer intensive mentoring and review panels.
"These and other approaches should be the focus of experimentation by states and districts.
-
Adopting an invalid teacher evaluation system and tying it to rewards and sanctions is likely to lead to inaccurate
personnel decisions and to demoralize teachers, causing talented teachers to avoid high-needs students and schools, or
to leave the profession entirely, and discouraging potentially effective teachers from entering it. Legislatures should not
mandate a test-based approach to teacher evaluation that is unproven and likely to harm not only teachers, but also the
children they instruct."
this is a report by eva baker, linda darling-hammond, and many others. necessary context for those newspaper articles.
http://epi.3cdn.net/b9667271ee6c154195_t9m6iij8k.pdf
"Nonetheless, there is broad agreement among statisticians, psychometricians, and economists that student test scores
alone are not sufficiently reliable and valid indicators of teacher effectiveness to be used in high-stakes personnel decisions,
even when the most sophisticated statistical applications such as value-added modeling are employed.
-
"For a variety of reasons, analyses of VAM results have led researchers to doubt whether the methodology can accurately
identify more and less effective teachers. VAM estimates have proven to be unstable across statistical models, years, and
classes that teachers teach. One study found that across five large urban districts, among teachers who were ranked in
the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third
moved all the way down to the bottom 40%. Another found that teachers’ effectiveness ratings in one year could only
predict from 4% to 16% of the variation in such ratings in the following year. Thus, a teacher who appears to be very
ineffective in one year might have a dramatically different result the following year. The same dramatic fluctuations were
found for teachers ranked at the bottom in the first year of analysis. This runs counter to most people’s notions that the true
quality of a teacher is likely to change very little over time and raises questions about whether what is measured is largely
a “teacher effect” or the effect of a wide variety of other factors...
-
"Evaluation by competent supervisors and peers, employing such approaches, should form the foundation of
teacher evaluation systems, with a supplemental role played by multiple measures of student learning gains that,
where appropriate, could include test scores. Some districts have found ways to identify, improve, and as necessary,
dismiss teachers using strategies like peer assistance and evaluation that offer intensive mentoring and review panels.
"These and other approaches should be the focus of experimentation by states and districts.
-
Adopting an invalid teacher evaluation system and tying it to rewards and sanctions is likely to lead to inaccurate
personnel decisions and to demoralize teachers, causing talented teachers to avoid high-needs students and schools, or
to leave the profession entirely, and discouraging potentially effective teachers from entering it. Legislatures should not
mandate a test-based approach to teacher evaluation that is unproven and likely to harm not only teachers, but also the
children they instruct."
Comments