Mind & Body

The Big Five Personality Traits May Not Be Reliable in Developing Countries

Scientific experiments are designed to extract a general truth from a specific sample — for example, if all 200 people in a study sneeze when they smell pepper, it's pretty safe to say that pepper has the potential to make any given person sneeze. To get a result that applies to the general population, though, it's best to use a sample that represents that population, and that's where many studies go wrong. Most scientists are at universities, after all, and university students are ready and willing to act as guinea pigs. Problem is, most people on Earth aren't university students, so when you try to apply those results to a different group of people, it doesn't always work. That's what an international team of researchers tried to do with one of the most influential measures of personality traits out there — and, yeah, it didn't work.

Biased Five

The personality measure the researchers tested is what's known as the Big Five model, so named because it measures people on five personality traits: openness, conscientiousness, extraversion, agreeableness, and neuroticism. Those five traits have been used in studies to analyze everything from people's leadership potential to their choice of partner — and the model has been used in developing countries to predict such high-stakes qualities as job performance and credit eligibility.

The problem, of course, is that the Big Five was originally tested in developed countries using participants who were mostly college grads, relatively wealthy, and white. That's a population known as WEIRD, for western, educated, industrialized, rich, and democratic. Studies in non-WEIRD countries started to suggest that maybe the Big Five wasn't as easy to generalize as scientists thought, so these researchers decided to take a deep dive into the data.

Opposites Attract

When we say deep, we mean it: The study, which was published last month in Science Advances, dove into four databases of personality data containing 31 datasets, which amounted to about 300,000 people from 30 different countries. Each of the databases looked at a slightly different slice of the world. The first was the World Bank's Skills Towards Unemployment and Productivity, or STEP, database, which involved face-to-face personality and cognitive tests with people from 14 countries. A second database included a hodgepodge of datasets from different researchers and institutions, so they used both face-to-face surveys and self-administered tests. The third was an online survey of people who lived in the same 14 countries from the STEP database, and the fourth was made up of self-administered surveys of people in the United States.

In essence, the Big Five personality test works because of something called factor analysis, which is a statistical method that takes a large number of variables and groups them into a limited number of "factors," or larger variables, that seem to explain each individual variable. If a lot of people take the same personality test, you can use factor analysis to tease out the few important traits that each answer maps onto — for example, a question about how much someone likes change would map onto openness, and a question about procrastination would map onto conscientiousness.

When the team completed the factor analysis for the Big Five in non-WEIRD countries and compared it to the data from the United States, there was a mismatch — make that a lot of mismatches. In many cases, the factor that a set of questions was supposed to measure actually mapped onto a completely different factor in a different country. In Ghana, for instance, all of the datasets showed that agreeableness could be explained by openness. Uh oh.

But not all was lost: The online results actually matched pretty well with the U.S. results, despite the fact that they were from more than a dozen different countries. The researchers think this might hint at one big reason why the other surveys didn't match: Maybe the people administering the face-to-face surveys influenced the answers. When they crunched the numbers, they found that individual "enumerators," as the survey administrators are called, could explain up to 25 percent of the variation in the data in some regions. The authors called that "worryingly high," and thought that the enumerators might be influencing the answers just by the way they asked the questions.

In addition, cultural differences could be at play. It's possible that participants just say what the enumerator wants to hear, which could explain why conscientiousness and agreeableness were the most problematic traits in the study.

Overall, the researchers said they couldn't point to one single thing that explains why the Big Five doesn't work as well in developing countries, but they have advice on how to make it better. They suggest using self-administered surveys whenever possible and randomizing the enumerators so they're not accidentally assigned to certain types of people. More generally, they suggest paying close attention to the quality of the translations and how each question is worded. But whatever we do, the researchers stress that we must improve the way we measure personality before we can actually understand the effects of personality traits in developing countries.

Get stories like this one in your inbox or your headphones: Sign up for our daily email and subscribe to the Curiosity Daily podcast.

Learn about more missteps in personality science in "The Personality Brokers: The Strange History of Myers-Briggs and the Birth of Personality Testing" by Merve Emre. The audiobook is free with an Audible trial. We handpick reading recommendations we think you may like. If you choose to make a purchase, Curiosity will get a share of the sale.

Written by Ashley Hamer August 12, 2019

Curiosity uses cookies to improve site performance, for analytics and for advertising. By continuing to use our site, you accept our use of cookies, our Privacy Policy and Terms of Use.