Comparison between treatment effects in a randomised controlled trial and an observational study using propensity scores in primary care

Beth L Stuart, Louise En Grebel, Christopher C Butler, Kerenza Hood, Theo J M Verheij, Paul Little

Although randomised controlled trials (RCTs) are considered 'gold standard' evidence, they are not always feasible or appropriate, and may represent a select population. Observational studies provide a useful alternative to enhance applicability, but results can be biased due to confounding. To explore the utility of propensity scores for causal inference in an observational study. Comparison of the effect of amoxicillin on key outcomes in an international RCT and observational study of lower respiratory tract infections. Propensity scores were calculated and applied as probability weights in the analyses. The adjusted results were compared with the effects reported in the RCT. Groups were well balanced in the RCT but significantly imbalanced in the observational study, with evidence of confounding by indication: patients receiving antibiotics tended to be older and more unwell at baseline consultation. In the trial duration of symptoms (hazard ratio 1.06, 95% CI = 0.96 to 1.18) and symptom severity (-0.07, 95% CI = -0.15 to 0.007) did not differ between groups. Weighting by propensity score in the observational study resulted in very similar estimates of effect: duration of symptoms (hazard ratio 1.06, 95% CI = 0.80 to 1.40) and difference for symptom severity (-0.07, 95% CI = -0.34 to 0.20). The observational study, after conditioning on propensity score, echoed the trial results. Provided that detailed information is available on potential sources of confounding, effects of interventions can probably be assessed reasonably well in observational datasets, allowing them to be more directly compared with the results of RCTs.