"Blinding" is not a silver bullet to deal with gender bias

A team of international researchers wanted to find out if there were biases in how managers evaluate their employees’ ideas. They conducted a field experiment in a large multinational technology company where they tested two different ways of evaluating ideas: one where managers did not know anything about the person who came up with the idea (what is known as “blinding”) and one where they did see the person’s name, which unit they worked for, and where they were located.

The results were surprising. The researchers found no bias against women and employees who did not work in the same location and unit as the evaluator.

What the researchers say: “Often, evaluators use the information they know about an idea proposer as a signal for idea quality. This may be reinforced when evaluators lack information, expertise, or resources to assess an idea’s details,” said the lead author. “Prior research shows that nepotism and hierarchy, for instance, can impact whether an idea is seen more favorably. Counterintuitively, our new findings clearly show that the gender or a shared unit or location of the evaluator has no impact on whether the idea is approved. It is clear that biases against these groups exist in other places but may not be universal.”

What can companies do in light of these findings? “Our research tells managers that simply hiding the identity of idea proposers, that is, what we call ‘blinding,’ is no silver bullet to improving idea evaluation,” he explained.

To conduct the study, the researchers used a blind evaluation tool, similar to those used in blind recruitment, blind auditions, and blind academic reviews, to remove any potential unconscious biases that evaluators might have. The logic was simple: is an idea evaluated differently when you can see the identity of the person who suggested it, as opposed to when you only see the idea but know nothing about its origin?

The researchers asked 38 innovation managers from a leading multinational company in the information and communication technology sector, all of whom had experience in evaluating early business ideas, to evaluate ideas proposed by other employees. Some ideas were displayed in a blind condition where evaluators were given no information on the idea proposer, while others were displayed in a non-blind condition with information on the employee behind the idea. Each innovation manager evaluated almost 50 different ideas, of which half were blind, and half were not. To ensure that the innovation managers acted candidly in their decision-making, they were unaware that the evaluation task was part of a wider experiment.

The researchers found that innovation managers provided the same evaluation score to ideas proposed by men and women, showcasing that the gender of the idea proposer had no impact on whether the innovation manager thought it was a good idea or not. The researchers also found that whether an evaluator shared the same unit and location as the proposer of the idea had no impact on the likelihood of the evaluator approving the idea.

The researchers suggest that these findings harm the business case for implementing blinding to erase biases. Although blinding ideas is technically straightforward and relatively cost-neutral to implement, missing out on blocked information can have some opportunity costs.

According to the researchers, the flip side of blinding is that it reduces the potential to connect employees with similar interests and learn from what other people are working on.

Given that evaluation is not an end goal for companies but one of many steps from an idea to a successful product, the researchers suggest that blinding is most helpful on a smaller scale to determine whether, how, and where biases exist before scaling any idea evaluation initiative within or across organizations. They encourage companies to carefully test and experiment with blinding to figure out how widespread biases are in their organization. Because biases are not as general as we may think, and clearly blinding is not the one-solution to improving idea evaluation.

So, what? This is a very, very interesting piece of research which upends a number of widely-held assumptions. It would be interesting to see if other researchers, studying other businesses, come to the same conclusions.