The Scientific Scourge Of Fake Data

Almost 10 years ago, a significant study on personal honesty was published. It indicated that simple method reduced lying by respondents who were filling out forms: if people signed an honesty declaration at the beginning of the form, rather than the end, they were supposed to be less likely to lie in their answers. The study was cited by other researchers and featured in a bestselling book written by one of its principal authors.

Now that study is being retracted. Over the years, efforts to replicate the results of the study have been unsuccessful, but now a more serious issue has been uncovered. Academics who took a close look at the underlying data cited in the study have determined that one of the main experiments cited in the study was faked, and that the data related to that experiment is fraudulent. The researchers who published the initial study agree and have asked the journal that published the initial study–the Proceedings of the National Academy of Sciences–to formally retract it.

It’s ironic that a study drawing conclusions about personal honesty would be based on fake data, but it’s the latest high-publicity example of a significant problem in the scientific community. Some have called it the “replication crisis.” We remember from our high school science classes that the scientific method involved developing a hypothesis, creating and conducting an experiment designed to test the hypothesis, describing the experiment and honestly publishing its results, and then letting the rest of the scientific community challenge the hypothesis, the experiment, and the data. The last step, in which other scientists played the role of skeptic and fact-checker and verifier by trying to replicate the experiment and test its results, was a key part of the whole process. And in the past, peer-reviewed journals played an important role in ensuring that the results of the experiments could, in fact, be faithfully replicated and the conclusions drawn were credible.

But something has obviously gone wrong, as a number of high-profile research findings can’t be replicated and there is increasing concern that data isn’t being collected or reported honestly or accurately. The “social sciences,” which encompasses the honesty study noted above, has been especially affected by the replication problem. And in the case of the honesty study, no one seems to know how the faked data was created in the first place. Four of the five authors of the study say they weren’t involved with collecting the false data, and the other one denies that he had anything to do with it. So, how did it happen, and why didn’t the initial authors of the study carefully review the faked data and question its bona fides before publishing the results? Some observers wonder if the behavioral studies that are now a staple of news feeds aren’t being influenced by the desire to create headlines and achieve clicks, leading researchers to overlook questionable data or methodologies.

You see signs these days that say that “science is real.” That’s obviously true, but the replication crisis demonstrates that not all scientific results are real. There’s nothing wrong with having a healthy skepticism about groundbreaking studies or sweeping pronouncements until after the underlying data has been thoroughly vetted and other researchers have replicated the results. As our high school science teachers instructed us, that’s what should have been happening in the first place.

My Overnight Brain

This morning, when I awoke, the Billy Joel song Honesty was firmly lodged in my brain — so firmly lodged that it kept playing and playing until I put on the iPod headphones and focused my brain instead on Television’s Marquee Moon.

Why did I wake up to Billy Joel’s plaintive anthem to forthright relationships?  I don’t have the slightest idea.  It’s not one of my all-time favorite songs.  I wasn’t humming it when I went to bed last night.  My best guess is that I heard it at some point over the last week or so, and while I slept my brain was simply shuffling through its catalog of recent stimuli, trying to figure out what to remember and what to discard.  It was just my good fortune to awaken when it was The Piano Man’s turn on the recently heard playlist.

I know my brain works hard while I sleep.  When I wake up, there often is a song, thought, or image at the forefront of my mind.  It’s not unusual for my overnight brain to help me with work issues.  If I am wrestling with how to structure an argument, I may hit the sack with the issue in mind and wake up with a carefully considered approach, down to the point of specific phrases or fully formed sentences that I can jot down and use.  On other occasions, I’ll open my eyes and immediately be confronted by a list of reminders and to-do items.  It’s as if the slumber of my conscious self frees my inner brain to grind away undisturbed, like a conscientious employee who is constantly distracted by a talkative boss and becomes truly productive only when the boss blessedly returns to his office.

I like the fact that my brain continues to work while I sleep.  I wouldn’t necessarily choose Billy Joel as my wake-up music, but it’s a small price to pay.