“When a measure becomes a target, it ceases to be a good measure.” Sociologist Donald Campbell and economist Charles Goodhart reached this conclusion some 40 years ago, after analyzing the measures affecting their own professions.
We’ve seen the Campbell and Goodhart law in action plenty of times. Bankers manipulated Libor, a benchmark for borrowing rates, to profit from trades. Businessmen use assorting accounting metrics to artificially inflate the value of their firms. Facebook has just admitted to overestimating how much video its users watch (paywall), meaning that advertisers have been overpaying.
Now it appears even scientists, among the most trusted professionals in the world, have fallen to abusing metrics. A new analysis of 60 years’ worth of studies in the behavioral sciences found that poor methods have flourished in academia because scientists, under pressure to publish to advance their careers, have focused on the metric of how many papers they put out.
There have been other bad practices too. Pushing negative results under the rug has created a skewed perception of our world. Lack of self-policing has helped high-profile scientific frauds to flourish. Cozying up with industry has biased our understanding of everything from prescription drugs to government policy.
As the number of scientists in the world has ballooned—from a mere 1 million in the 1940s to more than 60 million today—bad science has infiltrated every layer of the enterprise. Policymakers and politicians share the blame, for creating perverse incentives while pouring in ever greater sums of taxpayers’ money. But the buck stops with scientists.
Though they would hate to admit it, scientists are not unlike journalists. Both probe the world for truth, albeit at a different pace and with different methods; and both are under intense pressure to publish. And journalists are among the least trusted professionals. Scientists risk losing their credibility if they don’t mend their methods, and soon.
This was published as part of the Quartz Daily Brief’s weekend edition. Sign up for our newsletters, tailored for morning delivery in Asia, Europe and Africa, or the Americas.