Introduction
This section focuses on learning objective #3:
- Be able to describe and interpret common measures of effect used in clinical epidemiology
We focus on calculating and being able to apply:
- Relative risk ratio, relative risk reduction (or increase)
- Absolute risk and absolute risk difference
- Number needed to treat, number needed to harm
- Hazard ratio
- Odds ratio (optional)
How large is the treatment effect?
The data gained from randomized trials provides an estimate of the treatment effect. Knowing the size of the treatment effect informs treatment decisions, e.g.: Is the size of the effect large enough given the risks of adverse effects?
The size of a treatment effect can be represented in different ways.
- Relative measures
provide a ratio measure of the effect relative to control. Doesn’t take into account baseline risk; tends to be more stable across different populations.
- Absolute measures
provide a difference measure of the effect relative to control. Takes into account baseline risk; differs considerably across different populations.
Consider the following measures of effect from the Women’s Health Initiative Study ((2002))
WHI assessed the risks and benefits of long-term hormone replacement therapy in older women.
A key finding was that HRT increased rates of coronary heart disease in this group.
Relative risk: there was a 29% relative increase in coronary heart disease in the HRT group
Absolute risk: an additional 7 patients per 10,000 person-years suffered coronary heart disease in the HRT group
Number needed to harm: if you treat 1429 women with HRT rather than placebo for 1 year for one additional woman will suffer coronary heart disease
Importantly the above measures provided in relative and absolute terms are different ways to represent the same effect. Which of these do you think was reported in the media?
Measuring the effects of interventions
Consider a 2 x 2 table providing the results of a randomized trial:
| Endpoint | No endpoint | Total | |
|---|---|---|---|
| Treatment | a | b | (a + b) |
| Control | c | d | (c + d) |
Risk is another word for probability; in this context we also have incidence and event rate.
The risk of the endpoint in the treatment group, \(I_t\), is
\[I_t = \frac{a}{(a+b)}\]
The risk of the endpoint in the control (or placebo) group, \(I_c\), is
\[I_c = \frac{c}{(c+d)}\]
Relative risk is simply the ratio of these two risks:
\[RR = \frac{I_t}{I_c}\]
Relative risk reduction (or increase) is the increase or decrease in relative risk compared to 1. For example if \(I_c > I_t\), the relative risk reduction (\(RRR\)) is:
\[ RRR = 1 - RR \]
The absolute risk difference is the difference between the two risks. The absolute risk difference is typically presented as positive and described as either an absolute risk reduction or absolute risk increase as appropriate.
\[ARR = I_c - I_t \]
\[ARI = I_t - I_c\]
The number needed to treat (\(NNT\)) and/or harm (\(NNH\)) provide a way to express the absolute effect of the treatment. In the formula below \(ARR\) (\(ARI\)) is expressed as a percent.
\[ NNT = 100/ARR \]
\[ NNT = 100/ARI \]
Worked example: CURE
Recall the CURE Study PICO (The CURE Investigators 2001)
P: Patients within 24 hours of admission with acute coronary syndrome; average age approx. 64; multiple cardiovascular risk factors; patients with risk of bleeding excluded
I: Clopidogrel in addition to aspirin and other aspects of standard care
C: Aspirin and other aspects of standard care
O: Primary endpoint: cardiovascular death, nonfatal MI or stroke; important safety endpoint: major bleed
Results for the primary endpoint:
| Endpoint | No endpoint | Total | |
|---|---|---|---|
| Clopidogrel plus aspirin | 582 | 5677 | 6259 |
| Aspirin alone | 719 | 5584 | 6303 |
Measures of effect: CURE, benefits
- \(I_c\): 11.4% event rate in the patients randomised control (aspirin alone)
- \(I_t\): 9.3% event rate in the patients randomised intervention (aspirin plus clopidogrel).
- Absolute risk reduction: \(I_c - I_t = 2.1\%\)
- Relative risk: \(I_t/I_c = 0.82\) or 82%
- Relative risk reduction: \(1 - RR = 0.18\) or 18%
- Number needed to treat (NNT): \(100/ARR = 48\)
It is important to put the NNT into context:
You need to treat 48 high risk patients with acute coronary syndrome with a combination of aspirin and clopidogrel for 9 months rather than aspirin alone to reduce one additional event within the primary endpoint.
This puts the benefits of clopidogrel plus aspirin into perspective. It is also important to consider any key harms.
CURE Results, safety
Major bleeding was significantly more common in the clopidogrel group (3.7 percent in the clopidogrel group as compared with 2.7 percent in the placebo group; relative risk, 1.38; 95 percent confidence interval, 1.13 to 1.67; P = 0.001)
Measures of effect: CURE, safety
- \(I_c\): 2.7% event rate in the patients randomised control (aspirin alone)
- \(I_t\) 3.7% event rate in the patients randomised intervention (clopidogrel plus aspirin)
- Absolute risk increase: \(I_t - I_c = 1\%\)
- Relative risk: \(I_t/I_c = 3.7/2.7 = 1.4\) or 140%
- Relative risk increase: \(RR - 1 = 0.4\) or 40%
- Number needed to harm (NNH): \(100/ARI = 100\)
CURE: Bottom line
If you treat 100 high-risk ACS patients with a combination of aspirin and clopidogrel for 9 months rather than aspirin alone: there will be 2 fewer events from the primary endpoint (cardiovascular death, nonfatal MI or stroke) and 1 additional major bleed.
Hazard ratio
The hazard ratio is a ratio of the hazard rates. The hazard in question is often death or some other dichotomous endpoint. The hazard ratio is the ratio of the hazard rate in the treatment group against the hazard rate in control.
The hazard rate estimates the frequency of a hazard over time. An important difference between the hazard ratio and relative risk is that hazard rates (and therefore hazard ratios) take into account information regarding the time a participant is exposed to the drug.
The hazard rate uses the data observed in the trial to estimate the probability of a participant who is yet to experience the event, experiencing the event in the next time interval.
Hazard ratios come from ‘time-to-event’ (AKA ‘survival’) analyses. Time-to-event analyses are commonly depicted in Kaplan-Meier curves.
Due to the benefits of hazard ratios over relative risk, a lot of large clinical studies report hazard ratios for the primary endpoint. Even when the primary endpoint of a trial is a hazard ratio, sufficient information is usually provided to calculate the relative risk and absolute risk.
Hazard ratios and relative risk ratios are often relatively close to each other. One circumstance in which they will differ is in very long studies looking at mortality.
Sashegyi and Ferry (2017) provides a helpful discussion of the difference between hazard ratio and relative risk.
Task
- What will the relative risk of death be for a study that is more than 100 years long? Will the hazard ratio be the same?
Have a look at the abstracts of the following papers we looked at earlier. For each paper, determine the following
- Was there an effect on the primary endpoint?
- If yes, what was the size of the effect on the primary endpoint? Where appropriate present the outcome in terms of hazard ratio, absolute risk reduction/increase, relative risk ratio, relative risk reduction/increase?
- Zinman et al. (2015) Empagliflozin, Cardiovascular Outcomes, and Mortality in Type 2 Diabetes
- Rosenstock et al. (2018) Effect of ertugliflozin on glucose control, body weight, blood pressure and bone density in type 2 diabetes mellitus inadequately controlled on metformin monotherapy (VERTIS MET)
- SEARCH Collaborative Group (2010) Intensive lowering of LDL cholesterol with 80 mg versus 20 mg simvastatin daily in 12 064 survivors of myocardial infarction: a double-blind randomised trial
Odds ratio (optional)
Odds ratios are calculated when the groups in the study are defined according to disease or outcome rather than exposure. Odds ratios are important in case-control studies and in systematic reviews. In case-control studies the participants in the study are defined according to whether or not they are a case—i.e. whether they have the disease or outcome of interest—and the cases are compared to controls—participants who do not have the disease or outcome of interest.
| Case | Not case | |
|---|---|---|
| Treatment | a | b |
| Control | c | d |
| Total | (a + c) | (b + d) |
The odds (O) of an event is the probability of the event, \(p\) occurring divided by the probability of the event not occurring.
\[O = \frac{p}{1 - p}\]
An odds ratio (OR) is a ratio of two odds. In epidemiology, odds ratios are typically used with when we don’t have incidence data. This is the case when the groups have been defined according to disease rather than exposure as is the case in case-controlled studies.
The question here is what is the odds of begin exposed to the treatment among the cases, \(O_{case}\), and what are the odds of being exposed to the treatment among the non-cases, \(O_{noncase}\).
\[O_{case} = \frac{a/(a+c)}{c/(a+c)} = \frac{a}{c}\]
\[O_{noncase} = \frac{b/(b+d)}{d/(b+d)} = \frac{b}{d}\]
\[OR = \frac{O_{case}}{O_{noncase}} = \frac{ad}{bc}\]
Some things to note about odds ratios:
Odds ratios will be similar to relative risk when the risk of the event is low, but they come apart when the risk of the event is higher (it is easy to prove this using a 2 \(\times\) 2 table)
The odds ratio for exposure if you had an event relative to exposure if you did not have an event is the same as the odds ratio for the event given exposure, but it only makes sense to calculate this when you have incidence data (such as that given by a prospective cohort study or a randomized trial as we have here)
When you have incidence data it makes no sense to calculate an odds ratio: you can calculate the relative risk
Task (optional)
If you would like to practice this, have a look at the risk of bladder cancer in patients taking pioglitazone as reported in Piccinni et al. (2011), or the risk of acute coronary syndrome among patients taking both clopidogrel and proton pump inhibitors in the case-control section of Ho, Maddox, and Wang (2010).