If there’s anything we have learned from the pandemic, it’s that health information and misinformation are plentiful. We’ve also learned that health news can be incredibly confusing when it’s coming at us rapidly. How do we start to make sense of the health news articles we read, whether or not they’re about COVID, when we don’t have a scientific background? And how do we know when the information is coming from credible sources? Here are some of the tips I’ve recommended to students to help them make sense of and evaluate health news and information. These tips are applicable to all types of news and information, not just health news and information.
Making Sense of Health News from Credible Sources
If the news comes from a credible source, you still need to make sense of the information. Also, just because it’s coming from a credible source doesn’t mean you shouldn’t check its accuracy. Distilling complicated health information into news articles in the 24/7 news cycle presents challenges for relaying information accurately.
- Read beyond the headline and social media blurb. Catchy headlines and social media blurbs rarely capture the essence of an entire scientific article, and the wording of the headline or blurb may be misleading. The point of headlines and blurbs is to grab your attention, so they may be dramatic and shocking. Read the entire article to understand the topic.
- Go to the primary source. When an article cites a particular study or report, go back to that original (primary) source of information so you can read the information and draw your own conclusions. The news article is a secondary source; it reviews and interprets the original (primary) source. If you’re reading an article online, they may link directly to the primary source. If not, search for the source based on the information provided in the article.
- Understand the different types of evidence, and know that not all evidence is created equal. In science, there is the concept of the evidence-based pyramid. Scientists use this model to rank different types of evidence. While scientific studies should all be rigorous, the different rankings refer to how generalizable the results are. A study with a small sample of homogeneous participants can’t be generalized to the entire population. Also, animal studies are not yet generalizable to humans although they can provide valuable information about how to proceed with human studies. Unfortunately, the media tend to report all evidence as equal. Learn more about the evidence pyramid and the different types of evidence from this guide by Walden University.
- Beware of over-reliance on pre-prints. Pre-prints have been easily accessible for some time due to online repositories, but they were relied upon heavily in reporting on COVID because there was no other evidence. So, what is a pre-print? A pre-print is an article that has not yet undergone peer review (during the peer review process, other scientists evaluate the research to ensure the methods were applied correctly and the conclusions drawn can actually be drawn based on the study findings). Pre-prints may have studies that come from a high level on the evidence pyramid, which is good; however, until they have undergone rigorous peer review, they haven’t gone through the entire system of checks and balances put in place to ensure good science. The pre-print is likely undergoing peer review as you’re reading it, but the version you’re reading isn’t the final, reviewed version. And yes, the peer review and scientific publishing process does have its own issues, but that’s a topic for another post.
- Remember, scientists and journalists use different language. Scientists use hedging language; this is a concept called scientific tentativeness. Scientists say, “X may happen,” or, “X is unlikely to happen,” because they know that scientifically, there will be exceptions to the findings; X may happen in 99% of the cases, but if there is 1% of cases when X doesn’t happen, then ethically, they can’t say X will “always” happen. Scientists also speak tentatively because they understand that as science changes, information may change. Science should change. We don’t do the same things we did in the 1950s because we know more scientific information. In contrast to scientific tentativeness, journalists often use bolder, more absolute language. They may say, “X will happen,” or, “X will not happen.” Why do they use this language? Continue reading to learn the answer.
- Understand the different purposes and audiences. Scientists write for other experts in their field who are actively seeking out the research. They are writing strictly to inform others in the field, and scientists don’t have to worry about making the language understandable to non-scientists or capturing someone’s attention like journalists must do. Journalists must distill difficult scientific concepts down into something people without scientific backgrounds can understand, and they must do so in a way that grabs the attention of the audience and both entertains and informs them.
- Be aware of how the sample is being presented. We’ve heard things like, “90% of Americans approve of…,” and, “Two out of three dentists choose our toothpaste.” But what do these numbers actually mean? Also, how big was the sample size? 90% or two out of three of a small sample size (let’s say 100 people) is different from 90% or two out of three in a large sample size (let’s say 5,000,000 people). You want to look for the concrete numbers rather than probabilities and percentages. Also, if the news article is telling you something like, “90% of all Americans…,” you need to make sure the sample size was actually generalizable to all Americans. Just because 90% of the study respondents believed something doesn’t mean you can assume 90% of all Americans will believe that unless the sample was actually large enough and diverse enough to represent all Americans.
- Don’t forget that science uses specialized vocabulary. Specialized scientific vocabulary means words may not always mean what they appear to mean. This may be confusing, so seek clarification before making assumptions about information. For example, there is confusion right now about the efficacy vs the effectiveness of the coronavirus vaccines. Brueck (2021) does a great job of explaining this, and you should read her article for a more in-depth explanation; but basically, the efficacy is how well a vaccine works in clinical trials and the effectiveness is how well it works in real-life scenarios. Brueck also explains that a 95% efficacy rate does not mean 95% of the vaccine recipients will be protected from COVID while 5% will not be protected. Science and statistics are complicated which is why it is very difficult to distill these concepts down to manageable pieces of information for the general public.
Evaluating Information and Sources for Credibility
But what do you do when you come across a source you’re unfamiliar with? How do you know the information you’re finding is credible? You need to evaluate the source.
- Look for CRAAP. The first step many of us have been taught in school is the CRAAP test: currency (When was the information published?), relevancy (Is it applicable to your research need?), accuracy (Is the information correct?), authority (Is there an author that has proper credentials for this topic?), and purpose (What is the source trying to do – inform, persuade, etc.?). This test has become somewhat controversial because these cues encourage you to only look within the article (vertical reading), and when you do that, you can easily be fooled by something that appears to be credible but isn’t. CRAAP is a great starting point, but you need to take it further.
- Evaluate the CRAAP with lateral reading. In order to truly determine accuracy, authority, and purpose, you need to verify information with outside sources; to do this, open up browser tabs across your screen (laterally) and do some fact checking. Search for the author to see if they really have the credentials they claim to have, search for information on the topic to see if the information presented aligns with what is out there, look up any citations in the article to ensure they’re not fabricated and the information actually says what the author of the article you’re checking says (basically, go back to primary sources), and look up the site itself to determine which groups or institutions publish it and if those groups or institutions have a special interest in presenting information in a certain way. Check out this lateral reading video by John Green to learn more.
- Check a media bias chart. It’s no secret that some media outlets skew more heavily to the left of the political spectrum while some skew more heavily right and others are more center. The amount of skewing can also change depending on the medium (i.e. a news outlet’s website may be a bit more centered while their TV station may drift from the center). Unfortunately, this political skewing may influence how health news is presented. This interactive media bias chart will help you determine where some popular sources fall on the spectrum. Ideally, choose a source that is more to the center. If you don’t do that, at least try to balance your information by finding news from sources on different sides of the aisle.
- Check your own bias. When you seek out information, do you frequently use sources that align with your current beliefs, or do you seek out sources that challenge your beliefs? Most of us gravitate toward sources that align with our beliefs, especially when we are seeking out political news. When we do this, we are engaging in confirmation bias. The point of seeking out information is to find the correct (or the most correct) information, not to confirm your beliefs. We must evaluate all evidence equally and rigorously, but we tend to only be critical of information that doesn’t align with our beliefs. Check out this video to learn more about confirmation bias and how to overcome it.
- Critically evaluate images and videos. There was a time when “seeing was believing.” If we saw something, we could believe it was true. But then photoshop technology entered the scene, and now we can no longer trust that what we are seeing is real. Image manipulation has only become easier as technology progressed, and you often hold some basic image and video manipulation tools in the palm of your hand just by having a smartphone. Social media is flooded with images that have been manipulated, and we now have “deepfake” technology that allows you to swap faces and voices to make videos that look real but aren’t real. Check out the Pennsylvania College of Health Sciences Fake, Flawed, or Fairly Accurate guide for tips on how to critically evaluate images and video.
Librarians Can Help You
Evaluating sources is tricky, and it’s very easy to be fooled by information that looks and sounds accurate but isn’t completely accurate. You may find items online that are blatantly true or blatantly false, but remember that many times, things fall somewhere in the middle which makes it difficult to evaluate information. For example, a news article from a credible outlet has verifiable facts, but the presentation of the facts may make the conclusions misleading. This presentation of the facts may include opinion commentary by the journalist. If you’re having trouble evaluating news, ask a librarian for help. Your library (whether you’re going to a public library or to your school or university’s library) will likely have multiple options to get in touch with a librarian, whether you want to visit, call, email, or chat in real time. If you’re having trouble locating a library, let me know, and I can help.