[임상통계] DELTA2 guidance on choosing the target difference and ... reporting the sample size calculation for a RCT 리뷰

Tea_Tasting_Lady
2022-10-15
조회수 434

관련 LINK

논문 link

https://www.bmj.com/content/363/bmj.k3750

DELTA2 Project Link

https://www.ndorms.ox.ac.uk/csm/delta2-guidance


논문 소개

DELTA2 Guidance는 RCT(Randomised Clinical Trial) 임상 연구에서 목표하는 차이값(target difference)과 이에 따라 계산되는 표본 수 산출(sample size calculation)에 대한 Guidance로 University of Oxford의 Centre for Statistics in Medicine 에서 2019년에 최종 버전이 발표되었습니다. 이 논문에서는 이 Guidance의 주요 내용(target difference 설정 및 표본수 산출)에 대해 설명하고 있습니다. 


The DELTA2 project

  • the United Kingdom’s Medical Research Council/National Institute for Health Research Methodology Research Programme 의 지원을 받아 진행되었음

  • 기존 Guidance(Delta guidance)의 개정 (target difference, sample size calculation을 다룸)

    • on specifying and reporting the target difference (the effect size) 

      • in the sample size calculation of a randomised controlled trial.


Box 1: DELTA2 recommendations for researchers undertaking a sample size calculation and choosing the target difference  

  • Begin by searching for relevant literature to inform the specification of the target difference

  • Target difference을 찾기 위해 우선 문헌 고찰을 시작할 것

    • Relevant literature can: relate to a candidate primary outcome or the comparison of interest, and

      • inform what is an important or realistic difference 

        • for that outcome, comparison, and population.

  • Candidate primary outcomes should be considered in turn, and

  • 주 평가변수 선택지(Candidate primary outcomes)를 검토할 것 

    • the corresponding sample size explored. 

    • Where multiple candidate outcomes are considered, 

      • the choice of the primary outcome and target difference should be based on 

        • consideration of the views of relevant stakeholder groups (eg, patients), as well as 

        • the practicality of undertaking such a study with the required sample size. 

      • The choice should not be based solely on 

        • which outcome yields the minimum sample size. 

        • Ideally, the final sample size will be sufficient for all key outcomes,   

          • although this is not always practical. 

  • The importance of observing a particular magnitude of a difference in an outcome

  • 결과변수의 차이값 크기를 특정하는 것(Particular magnitude of a difference)은 중요함  

    • with the exception of mortality and other serious adverse events,

    • cannot be presumed to be self evident.

    • Therefore, the target difference for all other outcomes needs 

      • additional justification to infer importance to a stakeholder group.  

  • The target difference for a definitive trial (eg, phase III) should be one 

    • considered to be important to at least one key stakeholder group. 

  • The target difference does not necessarily have to be the minimum value 

  • 목표 차이값은 필요한 값의 최소값(the minimum value)보다 커도 무방

    • that would be considered important 

    • if a larger difference is considered a realistic possibility or would be necessary to alter practice.

  • Where additional research is needed to inform what would be an important difference

  • 추가 연구가 필요할 경우, anchor and opinion seeking method가 더 나은 방법임 

    • the anchor and opinion seeking methods are to be favoured. 

    • The distribution method should not be used.

    • Specifying the target difference based solely on a 

      • standardised effect size approach should be considered a last resort,

  • Where additional research is needed to inform what would be a realistic difference, 

    • the opinion seeking and the review of the evidence base methods are recommended. 

    • Pilot trials are typically too small to inform what would be a realistic difference and 

      • primarily address other aspects of trial design and conduct.  

  • Use existing studies to inform the value of key nuisance parameters

  • 기존 연구(existing studies) 결과치(e.g., 탐색 연구 pilot trial)를 표본수 계산에 활용할 것 

    • that are part of the sample size calculation. 

    • For example, a pilot trial can be used 

      • to inform the choice of the standard deviation value for a continuous outcome and 

      • the control group proportion for a binary outcome, 

      • along with other relevant inputs such as the amount of missing outcome data.

  • Sensitivity analyses,  used in the sample size calculation, should be carried out.

  • 표본수 계산에 사용된 값(e.g., the target difference, 대조군 반응율 등)들에 대한 민감도 분석(Sensitivity Analysis)을 수행해야 함 
    • which consider the effect of uncertainty around key inputs 

      • (eg, the target difference and the control group proportion for a binary outcome)

  • Specification of the sample size calculation, including the target difference,

  • 표본수 계산에 사용된 값들(e.g., target difference)은 양식에 따라 관련 문서(임상시험 계획서 등)에 보고되어야 함 
    • should be reported according to the guidance for reporting items (see table 1) 

      • when preparing key trial documents (grant applications, protocols, and result manuscripts).


The target difference and sample size calculations in randomised controlled trials 

RCT 연구에서의 목표 차이값과 표본수 산출 

  • The role of the sample size calculation is

  • 표본수 산출의 역할
    • to determine how many patients are required 

    • for the planned analysis of the primary outcome to be informative

    • It is typically achieved by 

      • specifying a target difference for the key (primary) outcome 

      • that can be reliably detected and the required sample size calculated

  • The precise research question that the trial is primarily set up to answer 

    • will determine what needs to be estimated in the planned primary analysis, 

    • which is known formally as the “estimand”

      • The target difference should be a difference that is appropriate for that estimand.

  • The target difference should be viewed as important by 

    • at least one (and preferably more) key stakeholder groups—  

      • that is, patients, health professionals, regulatory  agencies, and healthcare funders. 

    • In practice, the target difference is not always formally considered and  

      • in many cases appears, at least from trial reports, to be determined on convenience, the research budget, or some other informal basis.

  • The target difference can be expressed as an 

    • absolute difference 

      • (eg, mean   difference or difference in proportions) or 

    • relative difference 

      • (eg, hazard or risk ratio)

    • is also often referred to, rather imprecisely, as the trial “effect size

  • Statistical calculation of the sample size is far from an exact science

  • 통계적인 피험자수 산출의 한계 (가정 assumption이 사용되고 가정된 값의 차이에 민감)
    • Firstly, investigators typically make assumptions 

      • that are a simplification of the anticipated analysis. 

      • For example, the impact of adjusting for baseline factors is difficult to quantify upfront, 

        • and even though the analysis is intended to be an adjusted one

          •  (such as when randomisation has been stratified or minimised),

        • the sample size calculation is often conducted on the basis of an unadjusted analysis.

    • Secondly, the calculated sample size can be sensitive to the assumptions made in the calculations

      • a small change in one of the assumptions can lead  

        • to substantial change in the calculated sample size. 

      •  Often a simple formula can be used to calculate the required sample size.

  • it is necessary for researchers to balance

  • 통계적 가설 검정 setting에서 제1종 오류와 제2종 오류 사이에서 균형을 맞춰야 함  

    • the risk of incorrectly concluding that there is a difference (Type I error)

      • when no actual difference between the treatments exists, 

    • with the risk of failing to identify a meaningful treatment difference when the treatments do differ(Type II error)

    • Under the conventional approach, referred to as the statistical hypothesis testing framework  

      • the probabilities of these two errors are controlled by setting 

        • the significance level (type I error) and  

        • statistical power (1 minus type II error) at appropriate levels 

        • (typical values are two sided 5% significance and 80% or 90% power, respectively).

      • Once these two inputs have been set, the sample size can be determined given 

        • the magnitude of the between group difference in the outcome it is desired to detect   

          • (the target difference). 

      • The calculation (reflecting the intended analysis) is conventionally done 

        • on the basis of testing for a difference of any magnitude

  • A key question of interest is what magnitude of difference can be ruled out. 

    • The expected (predicted) width of the confidence interval can be determined 

      • for a given target difference and sample size calculation,   

    • The required sample size is very sensitive to the target difference. 

      • Under the conventional approach,  

        • halving the target difference quadruples the sample size for a two arm, 1:1, parallel group superiority trial  with a continuous outcome.

        •  차이값을 절반으로 설정하면, 표본수 산출 결과는 4배로 커짐(*아래 공식 참조) 

      • Appropriate sample size formulas vary depending on 

        • the proposed trial design and 

        • statistical analysis

  • In more complex scenarios, simulations can be used

  • 더 복잡한 연구 시나리오의 경우, Simulations도 사용 가능함    

  • It is prudent to undertake sensitivity calculations to assess

  • 가정값들 확인을 위해 민감도 분석, 계산 (sensitivity calculations) 을 하는 것이 현명함 

    • the potential effect of misspecification of key assumptions such as 

      • the control response rate for a binary outcome or 

      • the anticipated variance of a   continuous outcome 


Specifying the target difference for a randomised controlled trial

RCT 연구에서 목표 차이값 설정하기 

  • the specification of the target difference for a randomised controlled trial,   

    • a series of recommendations is provided in box 1 and table 1. 

    • Seven broad types of methods can be used 

      • to  justify the choice of a particular value as the target difference, which are summarised in box 2

  • Box 2: Methods that can help inform the choice of the target difference

    • Methods that inform what is an important difference

      • Anchor

        • using either a patients' or health professional’s judgment to define what an important difference is

        • by comparing a patients' health before and after treatment and then 

          • linking this change to participants who showed improvement or deterioration using a more familiar outcome

      • Distribution

        • determine a value based on distributional variation

        • use a value that is larger than the inherent imprecision in the measurement and therefore 

        • likely to represent a minimal level needed for a noticeable difference

      • Health economic

        • use the principles of economic evaluation

          • compare cost with 

            • health outcomes and

            • define a threshold value for the cost of a unit of healt effect that a decision maker is willing to pay

        • to estimate the overall incremental net benefit of one treatment versus the comparator

      • Standardised effect size

        • the magnitude of the effect on a standardised scale defines the value of the difference

        • For continuous outcome, the standardised difference can be used

          • e.g., Cohen’s d effect size

            • the mean difference/the S.D 

        • For binary or survival(time-to-event) outcome, odds, risk or hazard ratio can be used 

          • no widely recognised cutoff points exist 

    • Methods that inform what is a realistic difference

      • Pilot Study

        • to guide expectations and determine an appropriate target difference for the trial

        • Phase 2 study could be used to inform Phase 3 study

    • Methods that inform what is an important or a realistic difference

      • Opinion seeking

        • the target difference can be based on opinions elicited from health professionals, patients, or others

        • Possible approaches

          • forming a panel of experts

          • surveying the membership of a professional or patient body

          • intervewing individuals

      • Review of evidence base

        • the target difference can be derived from current evidence on the research question

        • Ideally, from a systematic review or meta-analysis of randomised controlled trials

        • In the absence of randomised evidence, evidence from observational studies could be used in a similar manner


Reporting the sample size calculation 

표본수 산출 결과를 보고할 때 포함되어야 할 사항들 

  • The approach taken to determine the sample size and the assumptions made should be clearly specified. 

    • all the inputs and formula or simulation results

      • so that it is clear what the sample size was based on.

      • allows the sample size calculation to be replicated, and 

      • clarifies the primary (statistical) aim of the study.

  • approach with a standard trial design (1:1 allocation,   two arm, parallel group, superiority design) and unadjusted statistical analysis, 

    • the core items are 

      • the primary outcome, the target difference appropriately specified according to 

        • the outcome type, 

        • the associated nuisance parameter 

          • (that is, a parameter that, together with the target difference, uniquely specifies the difference on the original outcome scale 

          • eg, the event rate in the control group for a binary primary outcome), and 

        • the statistical significance and power

  • More complicated designs can have additional inputs 

    • such as the intracluster correlation for a cluster randomised design

  • If the sample size is determined on the basis of a series of simulations,  

    • this method should be described in sufficient detail 

      • to provide an equivalent level of transparency and assessment


Discussion

논의

  • Researchers are faced with a number of difficult decisions when designing a randomised controlled trial, the most important decisions are

    • The choice of  trial design

    • primary outcome, and 

    • sample size 

  • The  sample size is largely driven by 

    • the choice of the target difference

  • The DELTA2 guidance provides help on 

    • specifying a target difference and 

    • undertaking and reporting the sample size calculation for a randomised controlled   trial. 

    • The guidance was developed in response to a growing recognition from funders, researchers, and   other key stakeholders (such as patients and the   respective clinical communities) of a 

      • real need for practical and accessible advice to inform a difficult decision. 

  • The key message for researchers is the need 

    • to be more explicit about the rationale and 

    • justification of the target difference 

      • when undertaking and reporting a sample size calculation.

    •  Increasing focus is being placed on the target difference 

      • in the clinical interpretation of the trial result, 

      • whether statistically significant or not.



0 0

Data Chef.

datachef00@gmail.com

ⓒ 2023 Data Chef.

Hosting by I'M Datachef

Data Chef.
e-Mail

datachef00@gmail.com


Seoul, Korea  ㅣ  Biz License 000-00-00000  ㅣ Hosting by Datachef.