Scientist can feel defensive when hard-earned data are questioned, so careful planning is required when approaching them with misgivings.
The scientific community is paying close attention to published work, and it means retractions are becoming more frequent, despite the careful pre-publication scrutiny. Retraction Watch publishes the depressing news of retractions almost daily. Although outright fabricated data is rare, mistakes do happen because scientists want to turn a blind eye to unwanted results, want to avoid being scooped or use inadequate experimental tools .
As a scientist, any work that you publish or data you collect becomes associated with your work or your lab. Thus, several reputations hang in the balance if retractions do occur. Rigorous experimental planning, data collection and analysis are paramount. But what happens if, for reasons outside of your control, these things don’t happen?
Unfortunately, early career researchers are often at the receiving end of non-reproducibility when moving into a new laboratory or taking on a new project. If you do believe that you’ve been caught up in a piece of work that doesn’t meet the high standards science demands, how do you approach your supervisor to voice your concerns?
With great care and consideration. That’s the advice that came out of a recent Reddit discussion on the topic.
If you believe your supervisor is the source of the problems, then you need to be able to predict his or her response and plan accordingly. More often than not, you’ll find that your PI would not question the need to correct the scientific record when convinced a problem exists. They, like you, believe in the integrity of science.
But anyone who identifies as a scientist can feel defensive when hard-earned data are questioned. Here are a few bits of advice to smooth a difficult conversation.
Make sure you’re sure. Double, triple, quadruple check the data, and then spend a night sleeping on it. If, after all that, you still think this concern is valid, then proceed.
Focus on the data. It’s easy to point the finger and assign blame to others, accusing them of manipulating data. This will only add fuel to the flames. Instead, look at what data you’ve been given and assess its quality. Note problems with what has been presented and left out, but avoid assumptions about researchers.
Verify your own data and those of others who are involved. If it means repeating experiments or spending some time examining the results from previous repetitions, so be it. You want to avoid going to your supervisor unprepared.
Prepare your results before approaching your PI to make sure that you’ve got your argument in order. In the case of missing data, one contributor to the Reddit conversation, PROFESSOR_LAVA_HOT, suggests making an “outline of all of the figures/data that needs to be included in the paper. Then place marks next to the ones that you actually have the data for (from you or the previous author), and the ones for which the data cannot be located.” This way you can clearly point out where missing data points are and why they concern you.
Be transparent. Honesty is always the best policy, whatever the situation might be. If it turns out that your concerns are not supported, then there isn’t anything to worry about. If, however, this isn’t the case, then at least you know that you’ve done the right thing.
If your supervisor does insist on sticking with the data and analysis that you’re unsure of, and you’re sure that you’re unsure, then you might consider asking to have your concerns made explicit in the publication.
If this isn’t an option, ask to have your name removed from the publication, making it clear that you do not want any involvement. Most universities and research institutions will have an ombuds office, where researchers can discuss their concerns confidentially. Worst case scenario, it might be wise to consider leaving the group and seeking another supervisor.
However difficult this conversation might seem, it’s considered good practice to be critical about data. Future scientific work that might depend on your results will be worthless if yours are too. If mistakes are made, it’s important to clear the record.
Too many papers are being published and retracted, and a reputation hangs in the balance every time. By keeping a level head and organising your thoughts and arguments in a concise way, discussions about data credibility can go smoothly.
Is this something you’ve experienced? How did you approach your PI? Do you have any further advice to add to ours? We’re keen to hear your stories so please do leave a comment below.
Further reading:
Robust research: Institutions must do their part for reproducibility
Recent comments on this blog
African astronomy and how one student broke into the field
From Doctorate to Data Science: A very short guide
Work/life balance: New definitions