What to Do When Two Studies Contradict Each Other in Your Literature Review (2026)
By Megan Carter 10 min read
When two studies in your literature review contradict each other, you do not pick a winner and quietly drop the other paper. You report both results, then explain the disagreement by naming what differs between the two studies: who was studied, how the outcome was measured, how each study was designed, and how much weight each result can carry. Those first three checks come straight from the Cochrane Handbook for Systematic Reviews of Interventions (version 6.5, 2024), which separates variability in participants and outcomes from variability in design and measurement in section 10.10.1. The disagreement is a finding in its own right, and writing it up as one is most of the difference between synthesis and a list of summaries.
What does it mean when two studies contradict each other?
Conflicting findings means two or more studies addressing the same question report results that point in different directions, or report effects of very different size. The Cochrane Handbook for Systematic Reviews of Interventions (version 6.5, 2024) splits the reasons into three kinds of variability in section 10.10.1: clinical diversity, "Variability in the participants, interventions and outcomes studied"; methodological diversity, "variability in study design, outcome measurement tools and risk of bias"; and statistical heterogeneity, "Variability in the intervention effects being evaluated in the different studies".
That three way split is the most useful thing a postgraduate student can borrow from systematic review methods, because it turns a vague worry into three concrete questions about the two papers in front of you. Two studies can disagree because they were not really asking the same question, because they were built and measured differently, or because of chance.
Source: Cochrane Handbook, chapter 10, section 10.10.1, read October 2026.
Do I have to decide which study is right?
No. You are not required to declare a winner, and in most literature reviews you cannot. The GRADE Handbook treats this situation as a judgement about the body of evidence rather than about one paper, calling it inconsistency, "an unexplained heterogeneity of results" (section 5.2.2).
What you must not do is settle it by counting papers. The Cochrane Handbook classes vote counting based on statistical significance as an unacceptable synthesis method, because such methods "all have serious limitations and can lead to the wrong conclusion", and because "underpowered studies that do not rule out clinically important effects are counted as not showing benefit" (section 12.2.2.1). The same chapter states that "Rules based on subjective decisions or statistical significance are problematic and should be avoided" (section 12.2.1.3).
One counting method the Cochrane Handbook does accept is narrower than it first looks: vote counting based on direction of effect, where "each effect estimate is first categorized as showing benefit or harm based on the observed direction of effect alone" and "Neither statistical significance nor the size of the effect are considered in the categorization" (section 12.2.1.3). Table 12.2.a records its limitation plainly, that it is "Less powerful than methods used to combine P values". So tally direction if you tally anything, say that is what you did, and do not present it as a result about significance.
Sources: GRADE Handbook, section 5.2.2; Cochrane Handbook, chapter 12, sections 12.2.1.3 and 12.2.2.1 and Table 12.2.a, read October 2026.
How do I tell a real disagreement from random noise?
Four signals, and the GRADE Handbook lists them together in section 5.2.2: the point estimates vary widely across studies, the confidence intervals show "Minimal or no overlap", the statistical test for heterogeneity returns a low p value (p <0.05), and the I² statistic is large. GRADE also notes a point that is easy to get wrong, that "direction of effect is not a criterion for inconsistency" on its own, since two studies can point opposite ways and still be compatible once the intervals are read.
Where a review reports I², the Cochrane Handbook gives a rough guide for meta-analyses of randomised trials in section 10.10.2:
| I² | What the Cochrane Handbook says |
|---|---|
| 0% to 40% | might not be important |
| 30% to 60% | may represent moderate heterogeneity |
| 50% to 90% | may represent substantial heterogeneity |
| 75% to 100% | considerable heterogeneity |
The bands overlap, and the same section warns that the thresholds can be misleading, since interpretation also depends on the magnitude and direction of effects and on the strength of the evidence for heterogeneity. For a narrative literature review, those bands are vocabulary for describing what a published meta-analysis found, not a test you run yourself. A low p value on the Chi² test, the handbook says in the same section, "provides evidence of heterogeneity of intervention effects (variation in effect estimates beyond chance)". That phrase, beyond chance, is the test to apply before calling a difference real.
Sources: GRADE Handbook, section 5.2.2; Cochrane Handbook, chapter 10, section 10.10.2, read October 2026.
How do I work out why two studies disagree?
Compare the two studies on four things, in this order, and write down what you find for each one. The first three are the Cochrane Handbook’s own categories from section 10.10.1, and the fourth catches what the first three miss.
- Participants, interventions and outcomes studied, which the Cochrane Handbook (version 6.5, 2024) calls clinical diversity: different ages, settings, countries or sampling frames produce different results from an identical design.
- The outcome measurement tool, which falls under what the same section calls methodological diversity, because two papers reporting "wellbeing" or "performance" may be measuring different constructs on different scales.
- Study design and risk of bias, also methodological diversity, and this is where a structured appraisal earns its place: a named checklist such as CASP gives you a defensible reason to say one result carries more weight, and the walk through is in our guide on how to critically appraise a research paper.
- Timing and analysis, including follow up length and how each set of authors handled missing data.
One of those four usually explains the gap, and the sentence you write names it. If none of them does, that is still a reportable result, and the honest phrasing is that the difference is not explained by the study characteristics available to you.
How do I write a paragraph about conflicting studies?
Use four moves in order: state the claim both studies speak to, report both results with citations, name the difference that plausibly explains the gap, then say what the disagreement means for your own study. The third move is the one that does the work, and the Cochrane Handbook (version 6.5, 2024) gives you the vocabulary for it in section 10.10.1: variability in participants, interventions and outcomes, or variability in design, measurement tools and risk of bias. Writing those four moves as four sentences produces a paragraph that reads as synthesis rather than summary, which is also the first problem named in 10 common literature review mistakes.
Here is the shape of a paragraph that makes all four moves. This is a format illustration with placeholder tokens, not a real result, and the author names, years and findings are deliberately empty:
Evidence on [construct] is mixed. (Author, Year) reports a positive association, while (Author, Year) reports none. The two studies measured [construct] with different instruments and recruited from different settings, which is the most plausible source of the difference. For the present study, [construct] is therefore measured with a single named instrument, and the comparison with [other setting] is treated as an open question rather than a settled one.
Litrevu is an AI research assistant that reads every paper a researcher adds, answers questions about them with a citation to the exact passage behind each claim, compares studies side by side, and checks the researcher’s own writing against their sources. When two of your papers disagree, you can ask it to compare them on participants, measures and design, up to 10 papers at a time, and each answer links to the passage it came from, which is the evidence your third move needs. The sentences in your chapter should be ones you wrote yourself.
How do I show the disagreement in a table?
Build a comparison table before you write the paragraph, because the table is what makes the explanation visible. The Cochrane Handbook for Systematic Reviews of Interventions (version 6.5, 2024) is unambiguous on this in the key points of chapter 12: "Tabulation and visual display of the results should always be presented alongside any synthesis".
Six columns do the job for conflicting findings: study and year, participants and setting, outcome and the instrument used to measure it, the direction of the result, the design and your risk of bias judgement, and the page or passage the result came from. That last column is the one people skip and then regret, because it is what lets you check a claim months later without rereading the whole paper. A free blank template, and a comparison built from up to 3 of your own PDFs with each filled cell quoting its passage, are in the Literature Review Matrix tool, and the longer extraction method is in our guide on how to extract and chart data from studies.
Source: Cochrane Handbook, chapter 12, key points, read October 2026.
What if my studies are qualitative and the findings do not line up?
Qualitative findings that do not line up are assessed, not averaged. GRADE-CERQual, the approach for judging confidence in findings from qualitative evidence synthesis, uses four components: "(1) methodological limitations, (2) coherence, (3) adequacy of data, and (4) relevance" (Implementation Science, 2018, DOI 10.1186/s13012-017-0688-3).
Coherence is the component that matters when accounts diverge, because it asks how clear and cogent the fit is between the data from the primary studies and the finding built from them, and the series’ paper on coherence names contradictory data as one of the things that weakens that fit. For a qualitative chapter, then, report the variation as a finding about conditions: this pattern appeared in these settings and with these participants, and did not appear in those. That is also how a divergent study becomes part of a theme rather than an awkward outlier, using the method in our guide on how to structure a literature review by theme.
Sources: Lewin and colleagues, Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series, Implementation Science 2018;13(Suppl 1):2; Colvin and colleagues, the series’ paper on coherence, Implementation Science 2018, read October 2026.
Which tools help me see whether papers agree or disagree?
Four tools and Litrevu cover different parts of this job, and as of October 2026 none of them does all of it. If what you need is papers you do not have yet, Elicit and Research Rabbit beat Litrevu outright, because Litrevu's only discovery is a basic open-access search and it works on the library you build.
| Tool | Best for | Works from | What you get | Where it beats Litrevu |
|---|---|---|---|---|
| Litrevu | Understanding, comparing and checking the papers you have already chosen | PDFs you add, plus open-access papers you pick from an OpenAlex search | Answers that cite the passage they came from, side-by-side comparison tables, and a check of your own writing against your papers | Only a basic open-access search, no screening at scale, no view on whether a source is contested |
| Elicit | Finding papers and extracting structured data from them | 138M paper corpus, and PDFs you upload | Reports, tables, summaries and structured comparisons | Far larger discovery corpus, systematic review screening into the thousands, purpose-built data extraction tables |
| Consensus | Answering an empirical yes or no question across the literature | 200M+ peer-reviewed papers | Synthesised answer with a meter showing how much of the literature agrees | Answers questions across all published work in seconds; nothing in Litrevu does this |
| Scite | Checking whether a paper has been supported or contradicted since | 1.6B+ classified citation statements | Citation contexts labelled supporting, contrasting or mentioning | Tells you if a source is contested. Litrevu has no view on this at all |
| Research Rabbit | Exploring outward from one paper to find related work | Citation graph | Visual networks of related papers, authors and topics | Visual citation-graph discovery; Litrevu offers nothing comparable |
The honest split: if you do not yet have your papers, Elicit, Consensus and Research Rabbit will get you there faster than Litrevu can. Litrevu's only discovery is an open-access search (OpenAlex) when you add sources, where you pick which papers to add; it cites only the papers in your library and does not search the web for citations. If you want to know whether a study you already cite has since been contradicted, Scite answers that and Litrevu does not.
Litrevu is an AI research assistant that reads every paper a researcher adds, answers questions about them with a citation to the exact passage behind each claim, compares studies side by side, and checks the researcher’s own writing against their sources. Elicit also works from uploaded PDFs, so the real difference there is the shape of the work: Elicit produces reports, tables and structured comparisons, with a purpose-built extraction workflow, while Litrevu is a conversation with your own papers in which each citation opens to its passage. Its comparison tables are built in chat for a handful of papers and are not an extraction workflow. Litrevu can also draft cited text into a document; many universities do not allow AI-written text in submitted work, even after you edit it, so treat that as notes and declare how you used it.
Comparison verified as of August 2026 against each vendor's own documentation. These products change quickly, so check the current feature list before deciding.
Litrevu is free to start with 25 messages and 2 drafted sections every 30 days, no card needed, and paid access is sold as time-limited passes rather than a monthly subscription, as of October 2026. Free tiers and prices for all five tools, read off each vendor’s own page, are compared in our guide to the cheapest literature review software for postgraduate students.
How do I report the disagreement so an examiner can check it?
Write down what you did, in the methods section, in two or three sentences. The SWiM reporting guideline, published in the BMJ on 16 January 2020 (BMJ 2020;368:l6890), sets out nine items for reviews that synthesise without meta-analysis, and three of them are exactly this task: item 1 covers "grouping studies for synthesis", item 3 asks authors to "describe the synthesis methods", and item 5 covers "investigation of heterogeneity in reported effects".
For a Masters or PhD literature review that is not a formal systematic review, those three items still translate cleanly: say how you grouped studies, say how you compared them, and say how you handled the cases where they disagreed. An examiner who can see those three sentences has no reason to suspect you ignored the awkward paper. The broader method for getting from a pile of papers to connected prose is in our guide on how to synthesize sources in a literature review.
Source: Campbell and colleagues, BMJ 2020;368:l6890, read October 2026.
Frequently asked questions
Should I leave out a study that contradicts everything else I found?
No. Leaving out a study because its result is inconvenient is selective reporting, and an examiner who knows the field will notice the gap. Include it, report what it found, and explain what differs about its participants, measures or design. A disagreement you have explained is stronger evidence of critical reading than a chapter where every paper agrees.
Does one contradicting study mean my research gap has gone?
Not usually. A single study pointing the other way often narrows a gap rather than closing it, especially if it used a different population or instrument from the rest of the literature. The GRADE Handbook's inconsistency criteria in section 5.2.2 are judgements about a body of evidence, which is why one paper rarely settles a question. State what the study found, then state which part of your gap it affects.
How many studies do I need before a disagreement is worth writing about?
Two. The GRADE Handbook's inconsistency criteria apply as soon as results come from more than one study, so a conflict between two papers is already reportable. What changes with more studies is the language you can justify: with three or four papers you can describe a pattern, and with a published meta-analysis you can quote its I² and heterogeneity test instead of characterising the spread yourself.
What do I write if I cannot work out why two studies disagree?
Say so, in those words. Report both results, list the study characteristics you compared, and state that the difference is not explained by the available information. That is defensible, and far better than implying a reason you cannot support. An unexplained disagreement is also a legitimate thing for a new study to address, so the sentence does double duty in your justification.
Can I use I² if I am not running a meta-analysis?
You can quote it, not calculate it. I² comes out of a meta-analysis, so in a narrative literature review you report the value a published review gives and interpret it with the Cochrane Handbook's bands from section 10.10.2. Those bands are published for meta-analyses of randomised trials, and the handbook warns the thresholds can be misleading, so quote the number with its source and avoid turning it into a verdict of your own.
Compare Your Own Papers Side by Side
Conflicting findings are slow work, because they need two papers held side by side at the level of the specific passage in each one. Litrevu reads the papers you add, answers questions about them with a citation that opens to the exact passage behind each claim, and maps where your papers agree and disagree, so the comparison table and the paragraph start from something traceable.
Litrevu can also draft cited text, and many universities do not allow AI-written text in submitted work even after editing, so check your institution’s policy, write your own text, and declare how you used any tool. You read every line and you check every citation. If you want to try it on your own papers, it is free to start with 25 messages and 2 drafted sections every 30 days, no card needed.
AI Research Assistant with Real Citations