How an Idea Becomes a Published Scientific Paper
How do you go from an idea to a scientific paper? Ask Science interviews exoplanetary scientist Moiya McTier to learn about the process and why we can trust scientific papers.
Sabrina Stierwalt, PhD
Listen
How an Idea Becomes a Published Scientific Paper
Today, I’m here with Moiya McTier, a researcher and PhD candidate at Columbia University. She was the first student to get degrees in both astrophysics and folklore and mythology at Harvard. She also runs workshops to help people think like a scientist.
Moiya is an expert in science communication as well as being a scientist herself, so today I want to ask her about the steps you have to go through to produce a scientific paper. How do you go from an idea to a published paper? Why should we trust these scientific publications more than other writing or news reports we find on the internet?
I’ve said on this show before not to trust articles that don’t cite their sources, but we’ve not yet talked about why we should trust those cited sources to begin with. So, thank you for being here, Moiya.
Thanks for inviting me on.
So, I get an idea, I write it down and then it’s published, right?
That’s the three step version, yeah. But there’s a lot more underneath the hood, otherwise it wouldn’t take five or six years to get a PhD, right? Starting from the beginning, people say that science starts with an idea, but it starts even before that. Because when I was starting grad school, one of the most intimidating things to me about the scientific process was coming up with an idea. Up until then, I had been doing research for a few years, but the research projects had always been handed to me and I’d never come up with my own idea or my own thesis for a research project.
People say that science starts with an idea, but it starts even before that.
You have to go through a lot of training to get to the point where you’re able to come up with your own idea or research project. It starts with an idea, but there’s a lot that goes into that part as well.
Could you maybe walk us through one of your recent projects?
Yeah, I’d love to. I actually just submitted for publication so this is the perfect time to talk about this. This idea actually wasn’t my own. One the faculty members in my department thought of this idea while he was in the shower. That’s where all the scientists do their best thinking!
So he thought about this idea and he brought it to me, and I thought it would be really exciting to explore. The idea that he had or the question that he wanted to answer was, ‘Are planets more likely to form around slow moving stars? Is there any sort of relationship between the speed of a star and whether or not it will host planets?’
I took that and ran with it. The thing I had to do next was figure out what type of data I was going to study to answer this question. As an astronomer, I have access to so much data. Observatories are taking something like petabytes or terabytes of data every year. So I have my choice of what type of data I want to use.
For this project, because I’m interested in planets and the speed of stars, I knew that I should probably use data from the Kepler mission which is responsible for finding most of the planets we’ve ever discovered and the Gaia mission which is responsible for giving us the most accurate and precise map of stars in the Milky Way Galaxy that we’ve ever seen before. So that was the next step: figuring out what type of data I wanted to use.
We talk a lot about the scientific method as scientists. It’s a good rule of thumb, it’s a good suggestion for how the scientific process goes, but it’s not exact.
And then I had to run experiments. We talk a lot about the scientific method as scientists. It’s a good rule of thumb, it’s a good suggestion for how the scientific process goes, but it’s not exact. Sometimes you’re going to deviate from the scientific method based on what type of research you do.
As an astronomer, I can’t really run experiments with stars. I can’t go out and have a control group and manipulate different stars with different methods to see how it changes the outcome which is what most people think of science. Instead I have to do all my experiments on a computer. I had to take the data that I had and split it up into different populations or different groups.
So I have a group of stars that have planets and a group of stars that don’t have planets and I can compare their velocities because that’s the question I want to answer. That sounds really easy, but it took me a few months to do this because I wanted to make sure I was doing it right. Which brings me to the next step that I did and the next reason you should really trust scientific publications.
One of the things I spent a long time doing was talking to other experts in the field and asking them what I could have done wrong. I went to different faculty members in my department, I emailed postdoctoral fellows who work in this field, and I said, I have observed this thing, I have this result, what do you think about it? Can you think of anything that I might have done wrong? Can you think of anything that might explain this that I haven’t thought of?
So, I really did do a lot of work trying to prove myself wrong, which might seem like it goes against what science is supposed to do, but a good scientist wants to make sure that her work is accurate, so I spent a lot of time proving myself wrong and eventually was successful in proving myself wrong … unfortunately, or fortunately depending on how you look at it.
So, I spent a couple of months looking at my results and thinking that planets are more likely to form around slow-moving stars or are at least more likely to survive around slow-moving stars. Then one day after long conversations with my advisor and with other experts in the field, I changed a couple of variables in my code and the result went away. This thing that I had been observing for months was actually just a result of some selection bias by the Kepler Telescope.
Can you tell us what a selection bias means in the instance of Kepler?
Yes, definitely. Kepler is an amazing telescope that’s not operating anymore. But for nine years it was staring out into space trying to find planets. And the way it did that was by measuring the amount of brightness that we get from different patches of the sky. If a planet passes in front of a star, then it will block some of that star’s light. Over time we would measure a dip in the amount of brightness we’re getting from a certain patch of sky.
But a telescope can only observe certain things or it will be limited by certain things. So Kepler can only observe stars that are bright enough for Kepler to see. So that’s a bias right there. Kepler is biased towards observing really bright stars.
It’s also going to be biased towards finding planets that are easy to see. Those will be planets that are close to their stars. Those will be planets that are bigger. It’s much easier to see a larger planet than it is to see a smaller planet. So that’s what I mean when I say selection bias. There are things that are easier for this telescope to see and so we are going to be biased towards seeing those types of things instead of things that are harder to observe.
I wanted this big flashy result about how planets are likely to form, but instead I got a result about how we have to be really careful when we analyze data because there are biases in our data that we might not be aware of.
What I was doing in this project was picking up on a selection bias of the Kepler telescope that people hadn’t really thought of much before. So I was really disappointed because I wanted this big flashy result about how planets are likely to form, but instead what I got was a result about how we have to be really careful when we analyze data because there are biases in our data that we might not be aware of. I think that’s a valuable lesson in and of itself.
Definitely. Sometimes these results are more important because they affect a lot more what we would do with the Kepler telescope, for example.
Exactly, and now people in the future who are analyzing Kepler data are aware— they can be aware of this bias. Which is how science works! We iterate on different methods and data and techniques, and we build knowledge slowly over time so that we can all hopefully, at the end of whatever this big process is, we can know … everything? I don’t think that’s true. To a point where we know everything.
So, you sought out an answer to whether or not fast versus slow stars have planets, and you find this bias that affects the answer that you were getting. Does this that mean you have answered that question definitively or is your answer instead that we can’t tell with the data we have?
A little bit of both. What I ended up saying in the paper was that we don’t see any evidence of a relationship between stellar velocity and planet occurrence around the Sun. The part of the Galaxy that the Sun sits in, we call that the solar neighborhood. So I say that we can’t say anything about this relationship in the solar neighborhood but conditions are different in different parts of the Galaxy. So maybe we can go analyze data from other parts of the Galaxy, maybe we can use different data sets. That’s why science is never done. You can always look under different conditions or you can use different types of data or different ways of analyzing the data.
That’s why science is never done. You can always look under different conditions or you can use different types of data or different ways of analyzing the data.
What I found was a very small piece to a very large puzzle. Which is the information about this one telescope was biased in this one way.
Our star obviously has planets. Is our star a fast- or a slow-moving star?
Our star is pretty average. In so many ways, the Sun is boring and not special. Our Sun is moving at 230 kilometers per second around the Milky Way Galaxy, which is 514,000 miles per hour. Our Sun is in a pretty boring part of the Milky Way. Our Milky Way— you can think of it as a pancake, and at the center there’s a big black hole. The Sun is about halfway between the black hole at the center and the very edge of the pancake that is the Milky Way Galaxy. So stars that are in different positions in the Galaxy will move at different speeds, and our Sun is pretty average.
Do you have advice for people who are interested in learning more about science and perusing the internet for whether it’s astronomy or not … How can we tell the difference between reliable sources of information, like articles written by people like you who have really done the background work to try to prove themselves wrong and look for all these potential biases, versus all the other misinformation that we can so readily find on the internet? How do we tell the difference?
It’s a really important skill to have in what I’ve heard some people call the misinformation age. With the internet, everything is at your fingertips, including lies. One thing that I always look for when I’m doing a cursory Google search is I look at the date of the publication. I’m much more likely to trust things that have been published recently than something that was published many years ago, for example. Not that they weren’t doing good work years ago, it’s just that things get updated, so I want to make sure I have the most recent version of an idea or a result.
With the internet, everything is at your fingertips, including lies.
Another thing is you want to make sure that the piece you’re reading says who the author is. If the piece you’re reading doesn’t say who wrote it, then to me that’s a really big red flag. Why does that person want to remain anonymous? Why don’t they want to make sure you can reach out to them and corroborate what they’re saying?
Those are two pretty easy ways to filter out things that won’t be reliable. Another way is that you should go to—and this is going to be kind of tautological—you should go to reliable sources. Things like academic journals have rigorous standards for what they publish. So, you know that if you’re reading something in an academic journal like The Astrophysical Journal or Science or Nature, they have rigorous processes to make sure they are not putting completely false things in their journals. So you can trust them. You can trust sources that cite those academic journals. The further removed you are from an academic source, the less trustworthy it is. Those are my rules of thumb.
Thank you for being here, Moiya, and telling us about your latest project and about how you take an idea to scientific publication. My listeners can hear more from Moiya by following her on Twitter and Instagram @goastromo and by subscribing to her mailing list at moiyamctier.com.
GET MORE Ask Science
You can become a fan of Ask Science on Facebook or follow me on Twitter. If you have a question that you’d like to see on a future episode, send me an email at everydayeinstein@quickanddirtytips.comcreate new email. Subscribe to QDT email newsletters to get science stories and much more delivered directly to your inbox. Listen and subscribe on Apple Podcasts, Spotify, Google Podcasts, Stitcher, or your favorite podcast app.
Image courtesy of shutterstock.
