International Comparison of Comparative Effectiveness Research in Five Jurisdictions

Adrian R. Levy, Craig Mitton, Karissa M. Johnston, Brian Harrigan, Andrew H. Briggs

Spurred by a desire to improve quality of care and to understand the relative value of medical treatments, there has been a recent surge of interest in publicly funded comparative effectiveness research (CER) in the US. As health technology assessment (HTA) shares some of the same goals as CER, and publicly funded HTA has been a feature within other industrialized countries for many years, a review of HTA activities in some of these countries can be a helpful source of information for the US debate. Informed by a literature review, and in two cases augmented by informant interviews, we reviewed the organization of HTA activities in five jurisdictions: Canada, Sweden, Scotland, the Netherlands and Australia. We provide a summary description of the healthcare system in each country as well as a description of the key features of their HTA bodies, with a particular focus on the processes of HTA for listing medications on public formularies. Four of the committees evaluating medications for formulary inclusion are funded by, but remain at arm's length from, the government (Canada, Australia, Sweden and Scotland), while the process is fully embedded within the government in the Netherlands. Each of these jurisdictions has a stated preference for comparative outcomes evidence from randomized controlled trials, but will, under certain circumstances, accept randomized evidence using surrogate markers, other comparators that are not directly relevant or non-randomized evidence. Health technology evaluation committees largely comprise health professionals, with public representatives included in the Canadian, Australian and Scottish committees. Scotland is the only one of the five jurisdictions reviewed to have industry representation on the evaluation committee. We identified seven characteristics that are shared across the jurisdictions reviewed and that potentially serve as insights for development of CER in the US: (i) the process must be responsive to stakeholders' interests, in that the turn-around time for assessments must be minimized, transparency must be maximized, the process must be considered fair using universally agreed standards and the process must be modifiable based on stakeholders' requirements; (ii) the assessment of medical technologies other than drugs may present different challenges and is managed separately in other HTA organizations; (iii) because of the link between HTA and reimbursement decisions, completion of the HTA process following regulatory approval can delay market access to new technologies, thus closer integration between regulatory approval and HTA processes is being explored internationally; (iv) there is a direct or indirect link to reimbursement in the jurisdictions explored - without this link the role of CER in the US will remain advisory; (v) each jurisdiction reviewed benefits from a single payer that is informed by the process - given the diverse multipayer environment in the US, CER in the US may usefully focus on generating comparative effectiveness evidence; (vi) a common metric for assessing intended and unintended effects of treatment allows comparison across different technologies; and (vii) one stated focus of CER is on therapeutic benefit among 'high-priority populations', including specific demographic groups (the elderly and children, racial and ethnic minorities) and individuals with disabilities, multiple chronic conditions and specific genomic factors. This will be difficult to achieve because epidemiological evidence of differences in therapeutic benefit among subgroups is detected through effect modification, or more specifically, statistical evidence of effect measure modification, typically on relative measures of effect. Few randomized trials have enough power to detect effect modification and these have been uncommon in the scientific literature. As consideration is given to the development of a publicly funded CER body in the US, much can be learned from the international experience. Nevertheless, there are some distinctive features of the US healthcare system that must be taken into account when assessing the transferability of these insights.