Core Outcome Measures in Effectiveness Trials

Core Outcome Measures in Effectiveness Trials
http://www.comet-initiative.org/home

Paula Williamson
MRC North West Hub for Trials Methodology Research

University of Liverpool

Acknowledgments

• Funding: MRC (MRP,) European Commission (FP7)

• COMET Management Group:
Doug Altman, Jane Blazeby, Mike Clarke

• COMET project coordinator: Elizabeth Gargon

• FP7 collaborators: Peter Tugwell, Maarten Boers,
Caroline Terwee, Holger Schunemann, Michael Rose,
Sunita Vohra, Roberto D’Amico, Lorenzo Moja

Why are outcomes important?

• Interventions are compared in RCTs by
measuring differences in patient outcomes
between the groups

• Selection of appropriate outcomes is crucial

• ‘Clinical trials are only as credible as their
outcomes’ (Tugwell 1993)

DMARD trials in RA

Completed Approved Never
application started
(not necessarily
meeting target Stopped
recruitment) early

Not Interim Other e.g. poor
submitted analysis
recruitment
Not Submitted
published Published

Missing Not Abstract Full
outcome accepted only publication
data
Some All
outcomes outcomes

Outcome reporting bias

• Selection of subset of original recorded outcomes, on the
basis of the results, for inclusion in publication

• Fully reported: OR 2.2 to 4.7 if statistically significant
(Dwan et al, PLoS ONE 2008)

• Reports vs protocols: 40–62% at least one primary
outcome changed, newly introduced or omitted

• Interviews with 59 trialists: lack of clarity about
importance and/or feasibility of outcomes chosen
(Smyth et al, BMJ 2010)

Core outcome set

• Agreed standardised set of most important outcomes

• The minimum – expect others to be collected

• Disease/condition specific
• All treatment types or a particular intervention
• Both benefits and harms

• COMET focus to date: effectiveness trials
• Relevant within routine clinical practice

Advantages of core outcome sets

• Increases consistency across trials

• Maximise potential for trial to contribute to
systematic reviews of these key outcomes

• Major reduction in selective reporting

• Much more likely to measure appropriate
outcomes

8

The COMET Initiative

• WHAT to measure

• HOW to measure a specific outcome
(validity, reliability, feasibility) – COSMIN,
PROMIS, http://proqolid.org/, TREAT-NMD

• Zarin et al, NEJM 2011 10

Cosmetic Outcomes Systematic Review
Aspects of cosmesis assessed

• www.omeract.org
• Trials 2007 8:38

COMET website

Jul-11 COMET website Jul-11
Aug-11 Aug-11
Sep-11 Cumulative Visits Sep-11
Oct-11 Oct-11
Nov-11 9000Nov-11
Dec-11 8000 Dec-11
Jan-12 7000 Jan-12
Feb-12 6000 Feb-12
Mar-12 5000 Mar-12
Apr-12 4000 Apr-12
May-12 3000 May-12
Jun-12 2000 Jun-12
1000
Jul-12 Jul-12
Aug-12 0 Aug-12
Sep-12 Sep-12
Cumulative Searches

1800
1600
1400
1200
1000

800
600
400
200

0

Development of core outcome sets: Issues
to consider (Williamson et al, Trials 2012)

• Scope
• Identifying existing knowledge
• Stakeholder involvement
• Consensus methods
• Achieving global consensus
• Regular review, feedback, updating
• Implementation
• Clear presentation

Appropriate outcomes

• Health service users seldom involved

• 20 studies (8 ongoing) include either patients,
caregivers and/or patient support group
representatives.

• Impact of patient involvement

– rheumatology (OMERACT): fatigue
– chronic pain (IMMPACT): expansion of previously

proposed core outcome domains
– paediatric asthma (parents and children)



Potential areas of bias

• Group composition

– Mainly researchers
– Patients/families seldom involved

• One panel or separate panels?
• How outcomes identified at start of process

– Mainly considered outcomes that were already
measured in clinical trials

– Seldom asked open questions

• Attrition of participants

Comparison of studies
(Sinha et al, Trials 2012)

COMET: What we are doing

• Development and maintenance of a publically
available searchable database of projects

• Systematic review
• Survey of COS developers

• Guidance: Developing core outcome sets
• Guidance: Integrating PROs into core outcome sets

• COMET III - 2013

Mental health

• Database: schizophrenia (published, review of
literature), forensic mental health (published, review
of literature and COS), outcomes for carers for
people with mental health problems (published,
review of instruments and carer opinion), bipolar
disorder (published, consensus conference)

• Non-database: offenders with mental health
problems, self harm

Neurology

• Database: epilepsy (ongoing, COS), Huntington’s disease
(ongoing, review of outcomes), Alzheimer’s disease
(ongoing, COS), Parkinson’s disease (ongoing, COS),
children with neurodisability (ongoing, PROMS), stroke
(ongoing, patient opinion), Guillain Barre syndrome
(published, ICF COS), multiple sclerosis (published, ICF
COS), non-epileptic seizures (published, consensus
workshop), peripheral neuropathy (published, consensus
workshop), West syndrome (published, COS), carpal
tunnel syndrome (published, review of outcomes)

• Non-database: dementia, HIV dementia

Developmental, psychosocial and
learning problems

• Database: neurodevelopmental delay (ongoing,
review of outcomes), autism spectrum disorders
(published, review of outcomes in intravenous
secretin studies; ongoing, COS)

Tobacco, drugs, & alcohol
dependence

• Database: smoking cessation (published, review of
outcomes, recommendations made)

How to measure recovery

• COSMIN group – www.cosmin.nl

• 408 systematic reviews of measurement properties
of health status measurement instruments

• Burgess et al (2011) – recovery measures in use in
Australian mental health services

• Cavelti (2012) – review of self-report instruments for
recovery from schizophrenia

COSMIN criteria for evaluation
of the quality of an instrument

Save the Date
Thursday 20th and Friday 21st June 2013

COMET 2013
The Midland

Hotel
Manchester, UK

Core Outcome Measures in Effectiveness Trials
http://www.comet-initiative.org/home