How to eliminate bias in performance rating

What’s the difference between a 10-point and a six-point performance rating scale, besides four points?

A lot, if you’re a woman being rated. A new study looking at student ratings of university teaching performance shows a substantial gender gap under a 10-point system disappears when the system used only has six points.

What the researchers say: “We were somewhat surprised at the magnitude of the effect,” says the lead author.

The researchers looked at what happened when a professional school at a large North American university switched from a 10-point to a six-point instructor rating scale.

Under the old system, male instructors garnered perfect 10 ratings 31.4 percent of the time in subjects with the highest numbers of male teachers. Female instructors got the top score in only 19.5 percent of cases. It was a different story though under the six-point scale. There, both sexes garnered top ranking almost equally; 41.2 percent for men and 41.7 percent for women.

The findings were replicated in a separate experiment where students were asked to read a transcript of a faculty lecture and then rate the lecture, with random variations made to the lecturer’s gender and the maximum allowable points.

Altering the ratings scale does not eliminate bias, the researchers caution. But the right one can limit the expression of pre-existing biases, such as stereotypes that associate brilliant performance with men more than women.

“Gender biases are particularly strong when people are making very fine distinctions between brilliant versus very good performance,” said the researchers. “With a six-point scale, we are basically taking away the ability to make those fine distinctions.”

“Making performance rating systems as fair as possible is important,” said the researchers, because professional advancement, from salary to seniority, can be significantly influenced by them. Any workplace concerned with eliminating gender inequalities should therefore pay attention to the design of the tools it uses to evaluate performance and merit, the study suggests.

So, what? Quite a few studies published recently in TR have shown performance ratings in almost any field are mostly useless. The benefit of this study is it demonstrates the more gradations you have in a performance rating, the more men will be favored. The reason? Especially in the professions, men are assumed to be superior. The more gradations, the more opportunities people have to demonstrate bias in their judgements.