Regulatory requirements regarding documentation for new medicines are constantly evolving. Previously, randomized controlled trials alone qualified as high-level evidence in pharmaceutical development — today, approval is often followed by requirements for additional post-approval studies or for so-called real-world evidence (RWE) studies. Here, I’ll explain these concepts and share epidemiological and statistical methods to tackle RWE challenges.
What is an RWE study?
There may not be any consensus regarding what exactly an RWE study is. However, I believe most people would agree that these are non-interventional studies (i.e., non-randomized trials) where patients are followed in the real clinical world, or, more specifically, where real-world data (RWD) are collected from various registries, such as claims databases or regional health registers. In other words, these are observational studies where patients, who are treated according to clinical practice rather than a clinical trial protocol, are followed using medical records and other registers. The questions studied cover a wide range of issues, from the effectiveness of a drug and patient profiling to health economics.
Why are RWE studies needed?
Some clinical questions simply cannot be answered by randomized trials. For example, it is not possible to randomize patients to smoking (unethical) or to certain risk groups (high BMI, high blood lipids, etc.). For this reason, observational/register studies have long been used in epidemiology to study risk factors for diseases and their outcomes. These types of research questions are no longer studied alone in academia; pharmaceutical companies often initialize studies that use register data to better map the risk factors of their patient groups, including identifying which subgroups to target for treatment (old or young, more or less sick, patients with certain comorbidities, or different genetic makeup, etc.). Furthermore, in some cases, randomized trials may be practically difficult to execute. For example, in studies of rare diseases, it may be difficult to find enough patients for a clinical trial. In this situation, a single-arm trial with an external comparator arm from RWD may be a good alternative.
Most important, RWE studies are used to complement clinical trials, where it is possible to answer questions that a clinical trial cannot address. How does a medicine perform in real clinical practice, where many patients change treatments several times? How does a treatment perform in an unselected patient group? How effective is the treatment in small subgroups (which requires a larger sample size to study than in clinical trial)? In a clinical trial, interim outcomes (for example, progression-free survival, PFS) are often studied as the follow-up time is too short (because of costs and practical reasons) to study mortality. What are the long-term side effects of a treatment? In addition, the larger study populations often provided by register studies are needed to detect any unusual side effects of a medicine. In an RWE study, it is often possible to have longer follow-up using historical data.
Another major advantage of an RWE study is that it permits studying patient groups excluded from the clinical trial, such as pregnant women, patients with multiple diseases, the elderly, or patients on other treatments. What is the treatment pattern of the patients, which drug is given as the first, second, or third line of treatment? What is the long-term survival rate for patients taking a particular medicine?
A health economic analysis is an important component of an RWE study. How much does the entire treatment cost? Can a new medicine reduce the number of side effects and comorbidities, thus reducing hospitalization and costs? How many high-quality years can a patient gain with a new medicine? And what are the cost savings on a societal level in terms of less sick leave and time spent by caregivers?
Why are RWE studies difficult to analyze?
In a clinical trial where treatment A is compared to treatment B, only randomization, i.e., a coin flip, determines which treatment the patient receives. The advantage of randomization is that the two treatment groups are more or less equal; groups A and B will have the same age distribution, the same severity of disease, and so on (unless chance plays a trick on us, or the study is too small). For example, if we learn that cancer drug A slows disease progression by 30 percent compared to drug B, we will tend to argue that this is due to drug A’s superiority, since the groups are comparable.
In normal clinical practice (the “real world”), things are different. It is the treating physician who, based on an assessment of the patient’s illness, general condition, other medications, etc., decides (in consultation with the patient) what treatment he or she should receive. Unfortunately, this often means that the groups receiving different medicines are not directly comparable, complicating the analysis. Another difficulty that often arises in RWE studies is that patients change treatments, for example, a patient may start treatment A, have an adverse reaction, and decide (in consultation with their physician) to switch to treatment B and then, after a certain period of time, try again with drug A (which is supposed to be more effective). Drawing conclusions about a given drug regarding PFS or survival is of course much more difficult in an RWE study than in a clinical trial, where patients (hopefully) follow a clinical protocol more rigorously during a predetermined follow-up period.
Epidemiological methods to tackle RWE challenges!
The existence of unequal background variables between groups being compared (drug A versus B) could (in very simplified epidemiological terms) be due to confounding. Fortunately, how to adjust for confounding variables in an observational (RWE) study has been one of epidemiology’s most researched areas for decades, and we are becoming quite good at it! Traditionally, the problem has been addressed statistically by using different types of stratification, standardization, and regression modelling. Basically, the groups are made comparable using mathematics. Nowadays, more modern study designs (target trial emulations to avoid various sources of bias) and methods derived from a branch of epidemiology called causal inference are often implemented. Using these methods (e.g., causal diagrams and various g-methods), the problem of confounding can often be almost eliminated or at least greatly reduced. Moreover, these models can also address time-varying confounding and exposure (e.g., patients changing treatment) by applying different weighting techniques. Several studies have shown that by applying the right methods in an RWE study, we can obtain estimates of the treatment effect that is equivalent to the unbiased causal effect in a corresponding randomized clinical trial.1,2,3
However, there is a problem with confounding adjustment in RWE studies: namely, that you can only adjust for what you have collected information on. So, for example, if we want to make two groups receiving different cancer drugs comparable in terms of the nature of their tumors (e.g., stages T, N, and M), we need to have that information in our register. If the information is missing, residual confounding will remain in our estimate of the treatment effect despite how modern and sophisticated our statistical model is, and we cannot adjust for it.
Residual confounding has meant that RWE studies are often met with skepticism. How can one be sure that all the necessary information has been collected to make the groups fully comparable? It is important to conduct a complementary quantitative bias analysis to understand the impact of an unmeasured confounder. For Swedish registers, for example, residual confounding is often less of a problem given their high quality and rich content, and they were often created for the purpose of conducting register research. In addition, the Swedish personal identity number system allows us to link registers nationally. Many other countries and regions have realized the importance of high-quality registers and claims databases and have invested significantly in registry infrastructure.
But regardless of the quality of the registers used in an RWE study, one must be careful when interpreting the results. By choosing the proper methodology and complementary sensitivity analyses, an estimation can come very close to the true effect. RWE studies therefore provide important information that complements the traditional randomized clinical trial!
Interested in learning more? Contact us!
Notes
1. Dickerman, B. A., García-Albéniz, X., Logan, R. W., et al. (2019). Avoidable flaws in observational analyses: An application to statins and cancer. Nature Medicine, 25(10). https://doi.org/10.1038/s41591-019-0597-x.
2. Wang, S. V., Schneeweiss, S., & RCT-DUPLICATE Initiative. (2023). Emulation of randomized clinical trials with nonrandomized database analyses results of 32 clinical trials. JAMA, 329(16). https://doi.org/10.1001/jama.2023.4221.
3. Matthews, A. A., Dahabreh, I. J., Frobert, O., et al. (2022). Benchmarking observational analyses before using them to address questions trials do not answer: An application to coronary thrombus aspiration. American Journal of Epidemiology, 191(9). https://doi.org/10.1093/aje/kwac098.
Read more from Perspectives on Enquiry & Evidence:
Sorry no results please clear the filters and try again
Synthetic Control Arms: Leveraging Real-World Data for Comparison in Single-Arm Trials
Data Challenges (and Solutions) for Externally Controlled Trials
Can RWE Help Restore Decades of Health Inequalities? Yes, and Here’s How
