[임상통계] DELTA2 guidance on choosing the target difference and ... reporting the sample size calculation for a RCT 리뷰
0
0
[임상통계] DELTA2 guidance on choosing the target difference and ... reporting the sample size calculation for a RCT 리뷰
관련 LINK
논문 link
https://www.bmj.com/content/363/bmj.k3750
DELTA2 Project Link
https://www.ndorms.ox.ac.uk/csm/delta2-guidance
논문 소개
DELTA2 Guidance는 RCT(Randomised Clinical Trial) 임상 연구에서 목표하는 차이값(target difference)과 이에 따라 계산되는 표본 수 산출(sample size calculation)에 대한 Guidance로 University of Oxford의 Centre for Statistics in Medicine 에서 2019년에 최종 버전이 발표되었습니다. 이 논문에서는 이 Guidance의 주요 내용(target difference 설정 및 표본수 산출)에 대해 설명하고 있습니다.
The DELTA2 project
the United Kingdom’s Medical Research Council/National Institute for Health Research Methodology Research Programme 의 지원을 받아 진행되었음
기존 Guidance(Delta guidance)의 개정 (target difference, sample size calculation을 다룸)
on specifying and reporting the target difference (the effect size)
in the sample size calculation of a randomised controlled trial.
Box 1: DELTA2 recommendations for researchers undertaking a sample size calculation and choosing the target difference
Begin by searching for relevant literature to inform the specification of the target difference
Target difference을 찾기 위해 우선 문헌 고찰을 시작할 것
Relevant literature can: relate to a candidate primary outcome or the comparison of interest, and
inform what is an important or realistic difference
for that outcome, comparison, and population.
Candidate primary outcomes should be considered in turn, and
주 평가변수 선택지(Candidate primary outcomes)를 검토할 것
the corresponding sample size explored.
Where multiple candidate outcomes are considered,
the choice of the primary outcome and target difference should be based on
consideration of the views of relevant stakeholder groups (eg, patients), as well as
the practicality of undertaking such a study with the required sample size.
The choice should not be based solely on
which outcome yields the minimum sample size.
Ideally, the final sample size will be sufficient for all key outcomes,
although this is not always practical.
The importance of observing a particular magnitude of a difference in an outcome
결과변수의 차이값 크기를 특정하는 것(Particular magnitude of a difference)은 중요함
with the exception of mortality and other serious adverse events,
cannot be presumed to be self evident.
Therefore, the target difference for all other outcomes needs
additional justification to infer importance to a stakeholder group.
The target difference for a definitive trial (eg, phase III) should be one
considered to be important to at least one key stakeholder group.
The target difference does not necessarily have to be the minimum value
목표 차이값은 필요한 값의 최소값(the minimum value)보다 커도 무방함
that would be considered important
if a larger difference is considered a realistic possibility or would be necessary to alter practice.
Where additional research is needed to inform what would be an important difference
추가 연구가 필요할 경우, anchor and opinion seeking method가 더 나은 방법임
the anchor and opinion seeking methods are to be favoured.
The distribution method should not be used.
The anchor and the distribution method 관련 link
https://hqlo.biomedcentral.com/articles/10.1186/s12955-018-1055-z
Specifying the target difference based solely on a
standardised effect size approach should be considered a last resort,
Where additional research is needed to inform what would be a realistic difference,
the opinion seeking and the review of the evidence base methods are recommended.
Pilot trials are typically too small to inform what would be a realistic difference and
primarily address other aspects of trial design and conduct.
Use existing studies to inform the value of key nuisance parameters
기존 연구(existing studies) 결과치(e.g., 탐색 연구 pilot trial)를 표본수 계산에 활용할 것
that are part of the sample size calculation.
For example, a pilot trial can be used
to inform the choice of the standard deviation value for a continuous outcome and
the control group proportion for a binary outcome,
along with other relevant inputs such as the amount of missing outcome data.
Sensitivity analyses, used in the sample size calculation, should be carried out.
which consider the effect of uncertainty around key inputs
(eg, the target difference and the control group proportion for a binary outcome)
Specification of the sample size calculation, including the target difference,
should be reported according to the guidance for reporting items (see table 1)
when preparing key trial documents (grant applications, protocols, and result manuscripts).
The target difference and sample size calculations in randomised controlled trials
RCT 연구에서의 목표 차이값과 표본수 산출
The role of the sample size calculation is
to determine how many patients are required
for the planned analysis of the primary outcome to be informative
It is typically achieved by
specifying a target difference for the key (primary) outcome
that can be reliably detected and the required sample size calculated
The precise research question that the trial is primarily set up to answer
will determine what needs to be estimated in the planned primary analysis,
which is known formally as the “estimand”
The target difference should be a difference that is appropriate for that estimand.
The target difference should be viewed as important by
at least one (and preferably more) key stakeholder groups—
that is, patients, health professionals, regulatory agencies, and healthcare funders.
In practice, the target difference is not always formally considered and
in many cases appears, at least from trial reports, to be determined on convenience, the research budget, or some other informal basis.
The target difference can be expressed as an
absolute difference
(eg, mean difference or difference in proportions) or
relative difference
(eg, hazard or risk ratio)
is also often referred to, rather imprecisely, as the trial “effect size
Statistical calculation of the sample size is far from an exact science
Firstly, investigators typically make assumptions
that are a simplification of the anticipated analysis.
For example, the impact of adjusting for baseline factors is difficult to quantify upfront,
and even though the analysis is intended to be an adjusted one
(such as when randomisation has been stratified or minimised),
the sample size calculation is often conducted on the basis of an unadjusted analysis.
Secondly, the calculated sample size can be sensitive to the assumptions made in the calculations
a small change in one of the assumptions can lead
to substantial change in the calculated sample size.
Often a simple formula can be used to calculate the required sample size.
it is necessary for researchers to balance
통계적 가설 검정 setting에서 제1종 오류와 제2종 오류 사이에서 균형을 맞춰야 함
the risk of incorrectly concluding that there is a difference (Type I error)
when no actual difference between the treatments exists,
with the risk of failing to identify a meaningful treatment difference when the treatments do differ(Type II error)
Under the conventional approach, referred to as the statistical hypothesis testing framework
the probabilities of these two errors are controlled by setting
the significance level (type I error) and
statistical power (1 minus type II error) at appropriate levels
(typical values are two sided 5% significance and 80% or 90% power, respectively).
Once these two inputs have been set, the sample size can be determined given
the magnitude of the between group difference in the outcome it is desired to detect
(the target difference).
The calculation (reflecting the intended analysis) is conventionally done
on the basis of testing for a difference of any magnitude
A key question of interest is what magnitude of difference can be ruled out.
The expected (predicted) width of the confidence interval can be determined
for a given target difference and sample size calculation,
The required sample size is very sensitive to the target difference.
Under the conventional approach,
halving the target difference quadruples the sample size for a two arm, 1:1, parallel group superiority trial with a continuous outcome.
차이값을 절반으로 설정하면, 표본수 산출 결과는 4배로 커짐(*아래 공식 참조)
Appropriate sample size formulas vary depending on
the proposed trial design and
statistical analysis
In more complex scenarios, simulations can be used
더 복잡한 연구 시나리오의 경우, Simulations도 사용 가능함
It is prudent to undertake sensitivity calculations to assess
가정값들 확인을 위해 민감도 분석, 계산 (sensitivity calculations) 을 하는 것이 현명함
the potential effect of misspecification of key assumptions such as
the control response rate for a binary outcome or
the anticipated variance of a continuous outcome
Specifying the target difference for a randomised controlled trial
RCT 연구에서 목표 차이값 설정하기
the specification of the target difference for a randomised controlled trial,
a series of recommendations is provided in box 1 and table 1.
Seven broad types of methods can be used
to justify the choice of a particular value as the target difference, which are summarised in box 2
Box 2: Methods that can help inform the choice of the target difference
Methods that inform what is an important difference
Anchor
using either a patients' or health professional’s judgment to define what an important difference is
by comparing a patients' health before and after treatment and then
linking this change to participants who showed improvement or deterioration using a more familiar outcome
Distribution
determine a value based on distributional variation
use a value that is larger than the inherent imprecision in the measurement and therefore
likely to represent a minimal level needed for a noticeable difference
Health economic
use the principles of economic evaluation
compare cost with
health outcomes and
define a threshold value for the cost of a unit of healt effect that a decision maker is willing to pay
to estimate the overall incremental net benefit of one treatment versus the comparator
Standardised effect size
the magnitude of the effect on a standardised scale defines the value of the difference
For continuous outcome, the standardised difference can be used
e.g., Cohen’s d effect size
the mean difference/the S.D
For binary or survival(time-to-event) outcome, odds, risk or hazard ratio can be used
no widely recognised cutoff points exist
Methods that inform what is a realistic difference
Pilot Study
to guide expectations and determine an appropriate target difference for the trial
Phase 2 study could be used to inform Phase 3 study
Methods that inform what is an important or a realistic difference
Opinion seeking
the target difference can be based on opinions elicited from health professionals, patients, or others
Possible approaches
forming a panel of experts
surveying the membership of a professional or patient body
intervewing individuals
Review of evidence base
the target difference can be derived from current evidence on the research question
Ideally, from a systematic review or meta-analysis of randomised controlled trials
In the absence of randomised evidence, evidence from observational studies could be used in a similar manner
Reporting the sample size calculation
표본수 산출 결과를 보고할 때 포함되어야 할 사항들
The approach taken to determine the sample size and the assumptions made should be clearly specified.
all the inputs and formula or simulation results,
so that it is clear what the sample size was based on.
allows the sample size calculation to be replicated, and
clarifies the primary (statistical) aim of the study.
approach with a standard trial design (1:1 allocation, two arm, parallel group, superiority design) and unadjusted statistical analysis,
the core items are
the primary outcome, the target difference appropriately specified according to
the outcome type,
the associated nuisance parameter
(that is, a parameter that, together with the target difference, uniquely specifies the difference on the original outcome scale
eg, the event rate in the control group for a binary primary outcome), and
the statistical significance and power
More complicated designs can have additional inputs
such as the intracluster correlation for a cluster randomised design
If the sample size is determined on the basis of a series of simulations,
this method should be described in sufficient detail
to provide an equivalent level of transparency and assessment
Discussion
논의
Researchers are faced with a number of difficult decisions when designing a randomised controlled trial, the most important decisions are
The choice of trial design,
primary outcome, and
sample size
The sample size is largely driven by
the choice of the target difference
The DELTA2 guidance provides help on
specifying a target difference and
undertaking and reporting the sample size calculation for a randomised controlled trial.
The guidance was developed in response to a growing recognition from funders, researchers, and other key stakeholders (such as patients and the respective clinical communities) of a
real need for practical and accessible advice to inform a difficult decision.
The key message for researchers is the need
to be more explicit about the rationale and
justification of the target difference
when undertaking and reporting a sample size calculation.
Increasing focus is being placed on the target difference
in the clinical interpretation of the trial result,
whether statistically significant or not.