Chattering Classes Meet The Psychometric Qualities of Performance Pay

I just read an excellent post on Bridging Differences about using test scores for teacher evaluation. My favorite part was this...

"And then there is the fundamental problem, as all psychometricians warn us, that tests should be used for the purpose for which they were intended, and not for other purposes. In other words, a test of fifth grade reading tests whether students in the fifth grade are able to read material appropriate for children their age. It cannot then be used to determine whether their teacher was good or bad."

Hallelujah
Testing validity depends on the test actually being used to measure what it was designed to assess.
I keep coming back to the idea that teachers don't go into education for the paycheck so why do, as Diane describes them, "the Chattering Classes" keep thinking that giving a mediocre teacher a couple extra bucks is going to make them a better teacher?

Tying student test scores to teacher pay is meant to increase teacher "performance" but what we are trying to do is increase student learning. If we had similar accountability to the banking industry we could still teach whatever wanted as long as we demonstrated the goal has been achieved ie "learning". But, we are more concerned about the test score than the learning that is meant to be demonstrated by the test. As an NBCT, I am totally for accountability and high levels of learning and teaching but until we develop a support structure for accomplished teaching from teacher prep to master teacher whatever we put in place will really evaluate how we support teachers not their effectiveness. Besides, there are plenty of economists who have studied how money is not an effective motivator in any profession and actually distracts from the goals of a profession. This approach focuses us on the gaming the system ala Enron and Bernie Madoff and not the goals the test scores are supposed to represent. Student Learning.

Maybe if the student test scores reflected actual learning instead of students ability to pass a test, teachers would be behind this type of reform. In the meantime, if we want to evaluate teaching lets start with evaluating their practice instead of how well their students can pass a test.

Supervision vs. Observation

I was asked recently about my experience of supervision in my first year. I am thinking back 12 years now. I was not observed formally more than three times my first year. I was observed informally numerous times. This year however, I have experienced a new type of observation from my supervisor. My current principal stops in almost every morning for about 30 seconds. He doesn't ask questions or interrupt. He just watches, smiles, and leaves.

As I have gained experience my perspective on this has changed. In the beginning I thought I was being "cheated" the necessary guidance I thought I needed. Over time I have discovered that some aspects of my practice are observed informally and frequently (lesson planning, management, and student rapport) while formal observations were almost incidental. I always tell new teachers that they are judged by how they walk their students down the hall and it is true. I have never received more than cursory feedback from formal observation although I have asked and even pressed administrators for feedback.

Recently, after reading Malcolm Gladwell's "Blink" which describes how experts and people in general can "thin slice" experiences and make the same or better qualitative judgments as scientifically based observations I decided that a principal can probably see "what a teacher actually does" as well or better in an informal 30 second observation as in a formal observation.

After having done some observation of student teachers, I have come to see the process of observation as much more fluid. The coaching or formal observation process can be extremely helpful to inexperienced teachers but I am not sure that it is the best use of administrators' expertise. I have always believed what my father said about management, that "It is easy to make someone do their job but hard to make them want to do their job." The observation/accountability process can intrude on this aspect of the administrator/teacher relationship.

I do think that observation is changing in the field but it has not happened in my school system yet. I think that observation for coaching is one area where teacher leadership can make a significant impact if the structure of schools is changed to allow for this type of leadership. Peer coaching may be more effective than supervision in supporting novice and struggling teachers. The evaluative aspect of administrator observations can actually distract from the goal of coaching. When a young teacher is observed for evaluation they are doing "their best" while, when they are observed for coaching it is understood that they are trying to improve and so may may show what is actually closer to their practice. Reflection on video is an excellent tool in this situation.

I have been trained as a coach using the Santa Cruz "New Teacher Center" protocols. They use a criterion referenced observational system that focuses on reflection as the primary tool for improving teaching practice.

If a supervisor is necessary to provoke reflective thought in a teacher maybe that teacher shouldn't be practicing or at least should still be in an apprentice position.

If I Were a Muppet

This is going to be a short and sweet post. Sometimes I feel like the male character in this classic bit from Sesame Street. I am really into education, teaching, the power of children, the amazing vision that children have and their ability to make us see the world different than we do. But, I get so into this whole teaching thing sometimes I forget where I am, I start passionately preaching to the choir until they remind me, I am not singing the same song anymore. So for all of you teacher leader freaks out there who absolutely just love education, this one's for you. With out us the profession may never start singing but, lets not get too far ahead of the parade that people don't know you are in it.

Dear Mr. Obama, Re: LDH

Dear Mr. President-Elect Obama,

I wanted to ask you to consider the rare opportunity you have to select Linda Darling-Hammond as Secretary of Education.
You see, the ED Sec position has always been a political decision, based on repaying the service of select individuals who have worked hard for a new president's campaign.

Here is a brief history of Secretaries of Education. It is from wikipedia so possibly incorrect, but it shows who held the position previously.

• Carter: Shirley Hufstedler had a distinguished career at the highest levels of legal and public service.
• Reagan: Terrel Bell, the impetus for a Nation at Risk, had a doctorate in education and had been a high school teacher as well as a superintendent.
• Reagan: William Bennet was an American neoconservative pundit, politician, and political theorist. He went on to become the "drug czar" for Bush 1.
• Reagan/Bush: Lauro Cavazos was the first Hispanic to serve in a cabinet post. He held B.A. and M.A. degrees in zoology from Texas Tech University, and a Ph.D. in physiology from Iowa State University. He was forced to resign as Education Secretary, amid an investigation into improper use of frequent flyer miles.
• Bush: Andrew Lamar Alexander (born July 3, 1940) is the senior United States Senator from Tennessee and Conference Chair of the Republican Party. He was previously the 45th Governor of Tennessee from 1979 to 1987.
• Clinton: Richard Riley Riley was elected governor of South Carolina in 1978. Riley made a huge impact by bringing the voices of teachers into the policy arena through teacher in residence program and supporting the establishment of the the National Board for Professional Teaching Standards.
• Bush II: Rod Paige began his career in college athletics. Later, as ED Sec Paige criticized the National Education Association (NEA) for obstructing "No Child Left Behind's historic education reforms," calling the NEA a "terrorist organization." He later said it "was an inappropriate choice of words" and apologized later the same day, but maintained that the NEA uses "obstructionist scare tactics" in opposing the law.
• Bush II: Margaret Spellings was the political director for Bush's first gubernatorial campaign in 1994, and later became a senior advisor to Bush during his term as Governor of Texas from 1995 to 2000.

The history of the position has seen only two ED Secs with actual experience in education. Only one, Terrel Bell, was a K-12 teacher. What did Terrel Bell do? He changed the future of education in America through bringing together countless stakeholders to create the National Commission on Excellence in Education, and publish a Nation at Risk. And he accomplished this while Ronald Reagan was trying to dissolve the USED and cut funding for all of its programs.

Darling-Hammond is a real threat to efforts to privatize education. She is a friend of unions because she believes in teachers, one the most leftist ideals you can maintain these days. One can hardly imagine what our school systems would be like if teachers were empowered to be the good guys. Mr. President give it a try and watch the inherent heroism of impassioned teachers transform out schools. Let a teacher lead us.

Please.

Gladwell vs. Willingham - Cage Match!

Its time for a Cage Match!
Good teaching is good teaching, but what does that mean?
Dear Dan,
You had me. I was on the road with you. Learning styles don't really exist, there are only learning preferences. But, in an amazing feat of under cutting of all your scientific jazzmatazz you said this:

"Good teaching is good teaching and teachers don't need to adjust their teaching to individual students learning styles."

I heard it. It planted itself like a seed. It began to grow. By Friday it was repeating itself, good teaching is good teaching, good teaching is good teaching, good teaching is good teaching, good teaching is good teaching, good teaching is good teaching, good teaching is good teaching, good teaching is good teaching.

What was it that bothered me about that statement. I couldn't remember. Then I saw this video of Malcolm Gladwell discussing the man who invented chunky spaghetti sauce, Howard Mouskewitz, one of his idols. It is a long video, so set aside almost twenty minutes in case you get engrossed.

What clicked for me was that when Mouskewitz conducted taste tests of 10 different tomato sauces approximately 1/3 of the participants preferred traditional, 1/3 zesty, and 1/3 chunky.

At the time that the testing was done, there were no chunky tomato sauces on the shelves. The people preferred something that was not even being offered at the time. The sauce companies all thought that good sauce was good sauce and that they didn't need to adjust their sauce to meet the preferences of their customers. When they did adjust and offer more variety, it took off, lots of people discovered chunky sauce and loved it.

So Dr. Willingham, here is my final answer, good teaching IS adjusting to individual students learning styles. When kids don't get something, you change your approach, you try a visual strategy, a kinesthetic strategy, a verbal strategy. It may not be that conscious, it may be a look in a kids eye, a shared experience, a connection between teacher and student that causes that teacher to try that strategy with that kid. It is a pragmatically scientific approach, not a positivist one and that is the reason it works. Because, in the classroom it is the learning that matters most, and figuring out why second.

Photo of Daniel Willingham by Dan Addison, University of Virginia Public Affairs.

Men and the Call to Teach

ABC news ran a story on the "Mistrusted Male Teacher" August 28th. It points out the inherent difficulties of being a male teacher in the early grades. As one of the fewest of the few, a male teacher in early childhood, I read the story with my own biases. Bryan Nelson, a 30 year veteran teacher summed up the lack of men in teaching this way,

"People don't think of men as caretaking or nurturing, which many of the young grades require," Nelson said. "And if you're a single man and you're going out to date somebody, when they ask you 'what do you do?' it just doesn't have the same cache as saying I'm an engineer or a scientist."

I haven't had a lot of problems with parents and issues of trust, but I have had my share. I once had a parent who told me to my face, "I wish my daughter had any other teacher than you." Some folks might have asked that the child be taken out of their room. I invited the parent to come in and watch what we do. ... every day. She came in for almost the entire year. I basically took her child from zero reading skills to reading in a year. We got along better and better over the course of the year. When it was all said and done, by the end of the year, she still didn't trust me. There was nothing I could do to change her opinion except continue to be open and honest with her. I have sometimes thought that maybe I was doing my boy students a disservice by being capable of teaching them as active boys only to send them to kindergarten to fail because they "couldn't sit still," or didn't seem to "pay attention." But, I also know that for some I have been their first introduction to what a caring man can be. I have even been able to help some kids with fathers because the fathers are more comfortable volunteering in my classroom. They see how they can care without being weak and nurture without giving up their masculinity.

Another recent article, Herland by L. J. WIlliamson, talks about this same issue. The author took a different point of view, sighting the lack of men in child care and teaching as contributing to keeping women, "in their place." She writes,

Of course, it won't be long before these kids get old enough to start noticing things like the fact that our country has never had a female president, or that male teachers typically don't come into their schooling until the academics get more serious. By that point, many will come away with the impression that caring for small children is women's work, and for the most part, they'll be right.

Also quoted in the article was this quote from William Marsiglio, Professor of Sociology at the University of Florida and author of Men On A Mission: Valuing Youth Work In Our Communities. He says,

"We pay our mechanics more to work on our cars than we pay people to watch our kids, which tells you something about our values. As long as we as society continue to perpetuate the narrative that being nurturing to children is inherently a gendered process, we'll never have gender equity. It will continue to be a society in which women are disadvantaged, and males are given an excuse not to be involved. "

I think the thing that bothered me about that ABC article is that it treated the chance of your child getting a "bad" male teacher or "good" male teacher as 50/50. Its just not like that. Most of the men who teach, especially in the younger grades, are teachers because they felt a call to teach. They aren't closet perverts or weak human beings. My greatest mentor was a male instructional assistant who also happened to be a minister. He felt a calling to teaching as he felt a calling to serve God. Perhaps if people thought of teaching the way that most teachers do there would be less fear. To most teachers, teaching is a calling, it is my calling, and it could be the calling of other young men, if they aren't pushed away from the field by the lack of social standing, pay, and support for being there.

Photo courtesy of: http://www.uppercasegallery.ca

To a Successful First Year Teacher

Q: When is a rookie not a rookie?
A: When they don't suffer from Pobrecito Syndrome.

Thanks to Vanessa at Pre-(K)Now's Inside Pre-K blog I finally have a vivid term: Pobrecito Syndrome, to describe when teachers feel sorry for at-risk students. This is a concept I was adamant about with the student teacher I supervised last year. She currently works as a 1st grade teacher in my school. I told her that she had to have high expectations for the students, don't pity them more than you expect them to succeed. If you do you fail and so do the kids.

This is what I told her when she asked me if she should come work at our school.

Our school is one of the 3-4 toughest elementary schools in the entire city. Possibly THE toughest. I am at this school for a reason. I am here to give the most help to the neediest children.

You were successful. Everyone (including principals from other schools and other local school systems) knows that if you can teach at here, you can tech anywhere.

Before our turn-around principal came the kids ran the school. 3 years ago, after spring break there were approximately 2-5 incidents of disruption every day. At least one false fire drill from students pulling the alarm per week.

The kids haven't changed, the school has.

I will give you my best advice, if you want to make a difference, if you think you can cut it, you should give it a try for at least 3 years. If I were you, I would commit totally to a certain amount of time, and then decide after that amount of time if I wanted to stay.

Most of all you have to realize that if you do come to here, you have to buy-in completely to the school and its culture. Don't do it if you think you need to come because you have to change the way things are done. You will only be disappointed. I have seen it many times with young teachers. You have to be comfortable with an authoritative environment. That is what our kids need because many of them don't have anyone in charge at home. They need to know they can't get away with stuff, it makes them feel safe, (and frustrated.)

Most of all I would do it while I was young, while I was still idealistic, while I had a great deal of energy. It is much easier to go from a hard school to an easy one than the other way around.
You will never be bored, you will always be challenged.

What ever you decide I *know* you will be successful and, in a couple years, you will be ready for a new challenge.(Maybe National Boards ;)

Best wishes in your decision. I hope I see you next year, if not, you will make a really great teacher for some really deserving kids.

Have a great summer and get lots of rest, you won't get much next fall now matter where you are.

She has done awesome this year. She totally has it down and her kids... are the best behaved on her grade level. She bought into the authoritative approach, which has got to be hard for a 23 year person, and she made it work. I find it fascinating how the authoritative approach translates into high expectations when it comes to at-risk students. I am so impressed by her I had to tell her in front of the entire blogosphere.

and the survey says .... EdSector doesn't get teachers

What do you mean teachers don't want to be financially rewarded by comparing their students to other teachers' students? A recent study by Education Sector and the FDR Group titled Waiting to be Won Over, surveyed 1,010 K–12 public school teachers and found:

Teachers are less likely today (than they were in 2003) to support paying teachers more based on test scores. Only half of teachers support the idea to measure teacher effectiveness based on student growth or "value added."

This finding struck me as convoluted because as a teacher I would not consider "student growth" and "value-added" synonymous. Student growth is what happens naturally when a student is supported and taught by a good teacher, but, it happens whether a teacher is good or not, it is natural. Value added is a measure of what the teacher does that makes a difference. It is not the norm, it is over and above "normal" student growth.

Below are the questions that I could find in the survey related to student growth and value-added. The number in front of the answers shows the percent by respondents.

How much would you favor or oppose giving financial incentives to each of the following: [Questions 19–23]

19. Teachers who consistently receive outstanding evaluations by their principals

24 Strongly Favor
34 Somewhat Favor
18 Somewhat Oppose
21 Strongly Oppose
3 Not Sure

To me this says that generally (58%) teachers trust the principal evaluation system to identify outstanding teachers.

20. Teachers whose kids routinely score higher than similar students on standardized tests
11 Strongly Favor
23 Somewhat Favor
25 Somewhat Oppose
39 Strongly Oppose
3 Not Sure

64% of teachers do not trust test scores as an evaluation system. This could be because teachers see test scores as invalid measures of their effectiveness not because they don't want to be held accountable.

21. Teachers who receive accreditation from the National Board for Professional Teaching Standards
25 Strongly Favor
40 Somewhat Favor
16 Somewhat Oppose
15 Strongly Oppose
4 Not Sure

This finding, not mentioned in the summary, shows that 65% of teachers trust National Board Certification as a valid measure of teacher effectiveness.

23. Teachers who work in tough neighborhoods with low-performing schools
34 Strongly Favor
46 Somewhat Favor
11 Somewhat Oppose
7 Strongly Oppose
3 Not Sure

Another finding not mentioned by EdSector, 80% of teachers understood the inherent complexity of teaching in tough schools and favor financial rewards for those teachers.

24. Suppose that in your district the students of some teachers make more academic progress—in terms of improved reading levels, teacher evaluations, and classroom tests—when compared to similar students taught by other teachers. How much would you favor or oppose financially rewarding those teachers?
10 Strongly Favor
34 Somewhat Favor
22 Somewhat Oppose
29 Strongly Oppose
5 Not Sure

This question asks a similar question to 19. concerning test scores. It runs into what I see as a philosophical wall because they both compare students to each other. This, I believe, for most teachers runs counter to our situational understanding of teaching. It expects teachers to trust the assumption that value-added statistical measures can accurately account for the differences between students including, geographic, financial, parenting, race, culture, ethnicity, language, family make-up etc. etc. This sort of "large" view of teaching does not honor what teachers are taught from the very beginning. Students are individuals, they have their own stories, problems, strengths, and that if we compare two different students to each other we do them both a disservice.

At your school, do you think there are outstanding teachers who deserve to be especially rewarded because they do a stellar job?
48 Yes
5 No
40 There are outstanding teachers, but I don’t think they should be especially rewarded
7 Not Sure

This question does not ask about "good" teachers. It asks about "stellar" teachers. There is a great deal of baggage associated with the "star" teacher in schools. They are often ostracized from the "group" because of a strange group mentality that tries to maintain the status quo. It happens in all parts of society but may be even more prevalent in teaching.

In the research I read recently most of the statisticians (see Doug Harris) stop short of using "value-added" measures for teacher evaluation but do support their use in school wide and district wide evaluation of schools' effectiveness.

My inferential findings are:
Teachers value principals opinions and the NBPTS. They don't value test scores and star teachers. There is something that many business folks and Wonks don't get about teaching. Outstanding teaching is not necessarily what should be rewarded (or punished). It is not about 1 teacher making a huge difference with 12 out of 18 kids and letting 3 kids slip because that is a good pass rate, it is about the teachers that consistently push kids to do their best. This may not show up in test scores, it may show up years later in a better society.

Don't wait up... I can not be won over to thinking about children or teachers as numbers.

to the art of teaching (teacher appreciation)

What we do is important. No one else will ever understand how many decisions we make over the course of a day. It must be hundreds. As I reflect each day I count the decisions like rocks weighing my soul. Was that the best response? Did I teach them? What did I do that made it better? The best thing about teaching is that you can get better at it but, only if you want to. So, here is to every teacher who tries to do it better than they did last time. And to every teacher who has inspired a child, even it it wasn't to become president, but may play an instrument, or like my daughter's Pre-K teacher did, inspire in my daughter a love of rocks. This is what gets at what we really do. Words like, standards, assessments, points, NCLB, don't mean much when you look at the world like teachers do. Teachers think in words like, inspire, learn, and wonder.

Here is to the artists of the field because teaching will never be a science, it is too big, wonderful and mysterious for that...

Teaching as an art

Art is about creating the remarkable from the ordinary. It is about raw materials, the history of the discipline, and the tireless pursuit, one might even say obsession, with making something remarkable. When I was a student in art school I gradually drifted towards creating room size sculptures called installations. I hoped that when someone entered an installation they would have an experience that changed them in some way.

To me, this is the definition of teaching. Teaching means to provide students’ experiences that change them in some way. For me it was a natural progression from creating installations to creating classroom environments in preschool. After all, the classroom is your most valuable tool for teaching young children.

Now, as a professional artist I find similarities in the process of teaching and painting. The aspect of teaching young children that is most like painting is the teaching of reading. My students come from poverty. They have very few prior experiences. When I paint I begin with the background, filling in the farthest horizon with aspects of the world that might not even be seen in the final painting. Then I begin to move forward, teaching letters and sounds in multiple ways. All the while I am focusing on these details I am building my students’ desire to read by bringing the literature alive. I am always thinking of the end result but responding to particular areas of a painting’s needs. With my students I am always monitoring the entire class’ progress while paying particular attention to the students the farthest behind and to the students who will light the fire of the desire to read. It is crucial in a painting to bring a painting forward in such a way that it seems like it has arrived, with its own logic and reality fully intact. Without considering the layers of paint under a representation of an object in a painting it is impossible create a painting that is true to itself. Without understanding the varieties and levels of experience a student has had before reading it is impossible to know what a student will understand about reading. When one or two students finally start to read and other students begin to see that they have the potential too it is like getting close to the completion of the painting. It is the most rewarding and the most important. Each mark on the canvas can make the painting better or set it back. It is the same with reading. As a student begins to read it is important that every advancement of their understanding is acknowledged and rewarded.

Teaching is an art because it requires a teacher to dig deep into their soul to find answers. It is not merely skill or technique, a conglomeration of lessons, charts, and assignments. It can best be described as knowing what to do when what you never imagined could happen, happens. It is important also to remember that the false dichotomy of art versus science is indeed false. Art and science are indeed two branches of the same tree of understanding. Each providing it’s unique light into reality.

Frank McCourt, in his memoir “Teacher Man”, describes just this type of situation. It happens on his first day, when a student throws a sandwich at another student and it lands on the floor of McCourt’s technical high school English classroom. If he were merely a technician he would mentally skim threw the training he received at teacher school, find the heading “What to do about flying food,” and perform the procedure. Of course that was not covered. He knew he did not have the students’ attention yet. He knew that his next move would decide his relationship with his students. He could have made the kid pick it up, he could have picked it up, he could have called the principal, he could have done a great many things but he knew he had to find the one thing that would accomplish his goal of creating a relationship with his students so he could teach. He had to innovate. So, he ate the sandwich. Was it the best move? That could be debated. The principal didn’t think so. But, it accomplished his goal. After he ate the sandwich he had them. They were putty, paint, clay, in his hands. He could teach. He was an artist.