The assimilation of evidence to understand a particular disease state, the burden of illness, epidemiology, competitors, and Health Technology Assessment (HTA) status is a fundamental element of market access preparation, and commonly involves a systematic literature review. While these reviews are essential for an effective market access strategy and in supporting HTAs, the approach can have limitations depending upon the knowledge of the disease area and the type of information the pharmaceutical company is trying to understand. From this perspective, there are roles for less rigorous approaches to assessing literature, and a more pragmatic ‘evidence review’, can sometimes be more appropriate as a first step. This article describes evidence reviews and systematic reviews, discusses their differences, and provides suggestions for when each one is most useful.
What are systematic reviews?
Systematic reviews are generally considered to be the highest quality of literature review due to their fully comprehensive and systematic nature; therefore they are usually essential for HTA. They provide an understanding of how new interventions compare against existing treatments in terms of clinical and cost-effectiveness for the same patient population, and capture all available data to answer a specific question with predefined criteria. Conducting a systematic review requires a reproducible protocol that should, in theory, allow any researcher to achieve the same results from a list of prespecified databases. These searches usually yield thousands of references which are then screened for eligibility according to the PICO (patient, intervention, comparator, and outcome) criteria by two independent reviewers. Any disputes are usually solved by discussion, or by the opinion of a third reviewer, but all references included within the systematic review results must adhere to the PICO criteria.
The selected references are read in detail, and relevant data are extracted into a spreadsheet so that studies can be easily compared for their key results, which are written up in a report that clearly states the methodology and dates of the searches. The systematic review results should include a PRISMA diagram, stating how many references were screened and excluded, and for what reasons exclusions were made. Comparisons can then be drawn between the results from the remaining references meeting the PICO criteria.
What are evidence reviews?
Evidence reviews are more flexible and holistic in nature than systematic reviews, and are not designed to answer such focused questions, but rather to understand where evidence exists, where it is lacking, and what this means for a product development strategy. They are bespoke to the needs of an intervention and can cover any and every aspect of a disease area, with a breadth of focus that could potentially far outreach a systematic review, including definitions and diagnosis, epidemiology, key clinical trials, economic evaluations, and previous HTAs.
Although the literature included within them is usually based on carefully considered PICO criteria, it is not necessary to develop strict protocols, search strings, or PRISMA. For this reason, they are not rigorous enough in their methodology to be used for HTA purposes, and should not be considered a substitute for systematic reviews in this regard. However, they can capture very large volumes of information in a relatively short period of time compared with systematic reviews.
SIRIUS have developed methods for evidence reviews which are efficient and publishable. The methodology involves initially using references identified from summary reviews, clinical guidelines, and HTAs, followed by supplementary searches in Embase and specific epidemiology databases to identify literature which is relevant and eligible for inclusion. As in systematic reviews, two researchers conduct all searches independently to ensure that no essential pieces of information have been overlooked, and the results are usually presented in a report. We have found that evidence reviews often capture at least as much information as systematic reviews, but in much less time.
While they are useful at all stages of the product development process, they are particularly useful before conducting a systematic review, helping to develop the PICO criteria for systematic reviews, identifying existing systematic reviews and gaps in the literature, and informing new HTAs with learnings from previous ones in the same patient population.
What are the advantages and limitations of each?
The advantage of systematic reviews is that they are designed to capture all published evidence relevant to a particular patient population, while the quality assessment process reduces potential bias, and thus enables statistical comparison between interventions across clinical trials. Evidence reviews, in contrast, have increased flexibility to include information from additional patient populations, which may be closely-related and still contain information that is important for an effective market access strategy, but failing to meet the strict PICO criteria of a systematic review.
Another advantage of systematic reviews is that the methodology for capturing literature is well-established, reproducible, and can be scrutinised by experts during the HTA process. While the evidence review methodology is publishable, and in our experience still captures all information of interest, it is not rigorous enough for HTAs, and should not be used instead of systematic reviews for such purposes.
The main limitation of systematic reviews is that their narrow scope and strict inclusion criteria may impede a holistic understanding of a disease area and be unsuitable for certain aspects such as epidemiology where published literature is often lacking or out of date. In such cases, the most up-to-date data can often be obtained through evidence reviews of registry databases where strict PICO criteria are unnecessary and sometimes detrimental to collecting all data of relevance.
An additional consideration is the considerable amount of time it takes to conduct a systematic review to answer a relatively narrow question. In general, it is recommended that a systematic review is conducted after an evidence review, thus capturing large volumes of information early on, in a relatively short amount of time. This data can then be used to inform a product’s market access strategy and develop a systematic review that will best reflect the benefits of the new treatment, while still meeting the rigorous requirements of the HTA.
In summary, both kinds of reviews have advantages and limitations, but are still highly valuable for informing a successful product development strategy and preparing for HTA. In general, SIRIUS recommend conducting an evidence review early on in the product development process, possibly during phase II or early in phase III clinical trials. The information obtained from an evidence review can then be used to inform focused systematic review questions and develop appropriate PICO criteria. Information from an evidence review can also be used for other activities in product development such as early engagement with horizon scanning organisations, publication strategies, and the preparation of a global value dossier.