Psychology, and psychometrics suffer from WEIRD science.

For decades, the consensus among psychologists (and those who frame psychometric tests) was that a cluster of five personality traits openness, conscientiousness, agreeableness, extraversion and neuroticism —or a slight variation thereof—universally defines the structure of human personality. However, when the team behind this research studied the Tsimane, an isolated indigenous population in the Bolivian Amazon, they found not five broad dimensions of personality, but two—prosociality and industriousness. Perhaps the Big Five aren’t so universal after all.

Perhaps what’s true for one culture cannot be extrapolated or mapped onto another. And this may also be true of individual and group experience and context.

What the researchers say: And that’s exactly their point in a new paper published in the Proceedings of the National Academy of Sciences. Sample diversity and socioecological theory, they argue are essential to improving the integrity and explanatory power of the social sciences. Among these psychology especially suffers from a sample diversity problem.

According to the lead author, even after repeated calls to diversify samples, over 90 percent of participants in studies published in the top developmental psychology journals still hail from North America and Europe, or as has been playfully (but aptly) termed, Western, educated, industrialized, rich democracies (WEIRD). This reliance on WEIRD samples is common to most subfields of psychology.

“And yet that’s only 18 percent of the world population today—and certainly unrepresentative of how we’ve lived over most of our species’ existence,” he said.

“Many areas of psychology if not directly, then implicitly, generalize study findings as human universals” he continued, “but if American students living in cities comprise our main study population, how can we generalize based on a tiny subset of humanity?”

Often those study populations are chosen out of convenience, but that choice can have a huge impact on the science they seek to inform, the researchers claim. “It’s easier and usually less expensive to study folks in your undergrad classes,” they said. “And people might argue that if they’re studying, say, perception, then maybe it doesn’t matter whether they do it here or someplace else. But it really does. There’s a growing set of examples of how certain stylized ‘truths’ once believed to be universal, are definitely not when you look in different places. And it’s not trivial.”

For the team, it comes down to embracing the same socioecological theory (also referred to as behavioral ecology) that has helped weave together the life and social sciences. Rooted in evolutionary and ecological principles, it’s often ignored in much of the psychological sciences. “Using that theory explicitly forces people to pay attention to how features of the physical and social environment might affect a whole litany of behaviors and psychological traits,” the lead author said. “The broad range of environments our species has experienced over its long history leads us to expect that many traits should vary as flexible responses to environmental conditions in ways that can lead to systematic differences across locales and situations and over time.”

So, what? To me, as both a psychologist and a behavioral neurogeneticist, all this is obvious—in fact scientists have long known that personality is not fixed but varies with physical and social context. A different culture would have different dominant personality traits. Within any major culture (even in a corporation) there are numerous sub-cultures which can each create separate dominant traits.

Personality varies with context, experience and situation as Daniel Goleman pointed out in his book “Social Intelligence.” You can be one kind of person in one social environment—or even conversation—and a quite different one in another. This makes a nonsense of most personality and psychometric tests.