It’s almost impossible to read every scientific paper that comes out these days. Nobody has time to screen databases searching for new and relevant studies for hours! Diligent choices are to be made to get the better knowledge on the market. Unfortunately, all papers aren’t born equal (quality speaking). So how can you choose the right ones? Here are a few tips.

1. The relevance of the subject

First criterion when selecting a study is the relevance of its theme. This is not only determined by the interest of the reader, although it is of utmost importance. It is also defined by its scope, by the proportion of professionals who encounter this particular theme in their daily clinical practice. If you’re a clinician, you probably want to use your knowledge for other purposes than making an impression at your family dinner. It has to actually help you, or even better, help your clients or your organization.

The more specific the subject, the narrower the public. New interventions or approaches tend to be in this category, but it is not a reason to put them aside. Specific subjects are often incredibly relevant. For example, what are the optimal exercise regimen parameters for a given condition?; what’s the most accurate test to help my diagnosis?; where should I refer my client next?; is this new modality safe and efficient? In addition, clinicians seem to like developing their expertise with specific clienteles and are more likely to seek knowledge that applies to a smaller proportion of the population. Relevance of the subject is therefore a subjective aspect that could vary with culture and position of the clinician which consumes the literature.

Evaluating the relevance of a subject also requires taking the context of the study into account. Can I compare it to the reality of my clients, my resources, the accessibility of services in my region or the organisation of my healthcare system? Knowledge can’t always be taken as read and generalized. It has to be taken within its context. An easy way to evaluate the context of a study is by asking where and when (ideally ≤10 years) it was held.

2. The type of article with the greatest value

Another important criterion for choosing a paper is its methodological quality. There are so many ways to conduct a research project, but those multiple options don’t all lead to the same credibility level of the results. The notion of level of evidence will be the subject of another whole blog publication, but we can still introduce here a few tips about the global quality of some types of studies. It is generally agreed that studies with the strongest methodological quality are randomized controlled trials, systematic reviews, meta-analysis and clinical practice guidelines. They can be seen as a pyramid, where randomized controlled trials are the strong base, leading to the summaries of knowledge that are the 3 others.

Although randomized controlled trials have a strong methodology, one study alone isn’t considered enough to draw solid conclusions on a given subject. In an ideal context, systematic reviews can be viewed as an inventory of the existing literature on a research problematic. The search strategy is rigorous and well documented. It includes consulted databases, inclusion and exclusion criteria, and a flowchart type of graph illustrating the articles’ retrieval process. This graph shows the reader when and why studies were excluded from the summary of knowledge. In a perfect world, systematic reviews would only contain randomized controlled trials of good quality, but it is rarely the case. Therefore, all systematic reviews aren’t considered to be of high level of evidence (basics of rating level of evidence of an article will be presented in a subsequent article).

A meta-analysis is like a level 2 systematic review. If the listed articles have similar methods and results, it is possible to pool the results and perform an analysis on them to increase the strength of the conclusions (because we all know we’re stronger together, right?). Meta-analyses will then be used to develop clinical practice guidelines in order to make recommendations (e.g., which test or intervention to use) and to judge the potential strength of these clinical recommendations. It is important to note that the strength of the clinical recommendations directly relies on the quality of the included studies at the very beginning of the process!

Be careful not to confuse clinical practice guidelines and expert consensus, which doesn’t have the same scientific value at all! Expert consensus, like expert opinions, often lack the scientific methodological rigor needed to be considered for clinical practice guidelines. In the same vein, narrative reviews are of lesser quality compared to a systematic review. Removing the rigor of the scientific process leads to papers too colored by the experience of the authors and can also be victim of confirmation (keeping only the information that validates their point) and desirability (keeping only the information that looks good to the public) bias. It could lead to thinking that a treatment works when it’s actually a natural evolution, forgetting to take the placebo effect into account or even failing to keep track of the times when the intervention did not work. To summarize in two words: rigor please.

3. Other few criteria determined by Bia’s team

We identified some other criteria that could influence a reader’s choice. You could also take these into account:

• The paper has been published in a peer-reviewed journal. This means that the content and structure of the study were validated before initial publication, which gives the article a minimum of credibility.
• The level of vocabulary used in the paper allows the reader to understand the key messages without being a specialist of the domain. If I understand what I read, chances are I will draw the right conclusion, apply the new knowledge correctly and explain it clearly to my target audience (e.g., my clients).
• The authors have several publications in renowned journals or have been cited several times.

Keep in mind that papers presenting null or non-statistically or clinically significant results are still of interest to work on reflexivity. If you want to know what works for your patients, you should also want to know what doesn’t work, and what has not been proven to work yet. To find studies saying what you want them to say is a trap (or a bias, in scientific language) that you need to avoid at all costs. Note that even if the methodology isn’t your favorite section of a paper, reading it is an investment to improve your critical thinking!

Have a good read!