What is the best way to evaluate teacher efficiency? After three years of research, the Bill and Melinda Gates Foundation thinks it has the answer. A balanced measuring system of test scores, classroom observations, and student surveys will do the trick, they report.
The Measures of Effective Teaching (MET) project, funded by the Bill and Melinda Gates Foundation and conducted out of the University of Michigan, released its final report last week after three years of study working with districts and nearly 3,000 teacher-volunteers on how to identify and promote effective teaching.
"By definition, teaching is effective when it enables student learning” they write and propose the question: “Can great teaching be measured?” After three years of study, observation, and collaboration with seven public school districts from Dallas to New York City, they conclude that yes, it can. Here is a breakdown of their key conclusions:
Great teaching, they believe, can be identified and identified best by equally using three evaluation measures: student surveys, classroom observations, and student test scores (which can count for 33 to 50 percent) as measures of student and teacher success.
“Each measure adds something of value,” the authors conclude. “Classroom observations provide rich feedback on practice. Student perception surveys provide a reliable indicator of the learning environment and give voice to the intended beneficiaries of instruction. Student learning gains (adjusted to account for differences among students) can help identify groups of teachers who, by virtue of their instruction, are helping students learn more.”
Their conclusion that test scores could effectively either hold equal weight (33 percent-33 percent-33 percent) with student surveys and classroom observation scores in teacher evaluations or up to 50 percent of the evaluations (with the surveys and classroom observations each counting for 25 percent) comes as a result of information they collected from a unique experiment they conducted for the project.
It was from analyzing the results of this experiment, in fact, that they believed they could really make the claim that "teacher effectiveness can be measured:" they took higher-performing teacher-volunteers and lower-performing teacher-volunteers and assigned them to a random classroom for a year. They found from this study that the students in classes with higher-performing teachers measurably learned and achieved the most (regardless of previous scores), and the students in the classes with lower-performing teachers learned and achieved less. It was less about the student's demographic, they discovered, and more about the teacher; the findings enforced their conclusion that although evaluations are most reliable when balanced by several measures, student test score data was significant in indicating a teacher's current and future potential for success.
"We didn't know if in fact what we were seeing -- the differences we were seeing between teachers were about the teacher or about the students who were coming into their class," Steve Cantrell, Gates' chief education researcher, told The Huffington Post. "By randomly assigning students to teachers, we were able to show that teacher effectiveness is really about the teachers."
However, the authors note that "the vast majority of teachers are in the middle of the scale, with small differences in scores producing large changes in percentile rankings." They also found that despite the best teachers doing well in all classroom, new teachers and those in the middle often do not have the support or resources to improve.
“Teachers want to succeed, but they typically lack the conditions for success,” the authors wrote. “Teachers generally work in isolation. They’re on their own to adjust practice to better serve students. What guidance they get often is plagued by vague teaching standards, overly numerous and often trivial learning objectives, and testing systems that measure only some of the outcomes that educators value for students.”
In response, the authors gave three key suggestions for districts on constructing effective evaluation systems:
For these systems to work, the authors explain, there need to be the right measures, the right measurement processes, strong communications, and an awareness of how information can be distorted. Once these are implemented, efforts and expectations for teaching can be aligned with what is actually taking place in the classroom.
“If we want students to learn more, teachers must become students of their own teaching. They need to see their own teaching in a new light. Public school systems across the country have been re-thinking how they describe instructional excellence and let teachers know when they’ve achieved it,” said Tom Kane, Professor of Education and Economics at Harvard’s Graduate School of Education and leader of the MET project. “This is not about accountability. It’s about providing the feedback every professional needs to strive towards excellence.”
Jay P. Green, however, a professor of Education Policy at the University of Arkansas, called the Gates research a "political document and not a research document."
Others have voiced their concerns, as well. Special education teachers, for example, and teachers of the arts, drama, music, and other arguably less-measurable classes are worried about how they will be evaluated.
"When you can come up with a standardized child raised in a standardized home, then perhaps a standardized test is appropriate. Until then, I'm not seeing any way to compare the job done by the CD special education teacher (those who work with the kids with the most extreme disabilities) and the teacher that is teaching a class of AP physics to college bound kids," one teacher said in response to the report.
Another teacher explained that they had once overheard students even plotting to "fail a test" so they could get their teacher fired.
President Randi Weingarten of the American Federation of Teachers Union, however, who had previously been critical of the MET Project's teacher quality research, released a statement in approval of MET’s final conclusions. "The MET findings reinforce the importance of evaluating teachers based on a balance of multiple measures of teaching effectiveness, in contrast to the limitations of focusing on student test scores, value-added scores or any other single measure," she said.
In addition to the reports, the MET project is also working on putting together an online video "Library of Practice" with the help of more than 300 great teachers recording over 50 lessons. They plan to make the video library available to districts and states by the end of the 2012-13 school year. Videos of all the 3,000 teacher-volunteers will also be made available for study to researchers and education schools.
Ryan Kinser, a MET teacher-volunteer and eighth grade English teacher who is rated "highly effective" by his district at Walker Middle School in Fla., said that when viewing one of his 10 minute lectures on the meaning of "hierarchy" during his participation in the project, he saw that students seemed bored and noticed that he "looked wooden" and "talked too much."
"It forced me to reflect and better prepare for my kids," Mr. Kinser told The Wall Street Journal about the experience.
“There is a great potential in using video for teacher feedback and for the training and assessment of observers,” the authors of the MET report wrote.
During their research, the authors found that most teachers distrusted the current systems of evaluation in place in their districts. Whether or not the MET project's findings will work or be implemented and implemented well into the chaotic world of measures in Education, the authors underline that "implementing specific [reliable] procedures in evaluation systems can increase trust in the data and the results."
"Evaluation has become bigger than learning. Who is evaluating those who are evaluating?" one teacher voiced his concern. "The bricks were suppose to support the structure / than all the rooms were filled with bricks / and there was no space for learning and curiosity," he wrote. Many believe that at least two of the most important aspects of great teaching - empathy and creativity - are not included in these evaluations and are perhaps immeasurable, in fact.
Janel Spencer is a writer and content editor for 360 Education Solutions