ISPOR US REPORT 2023

Alternative Network Meta-Analysis Methods

At ISPOR International 2023, there was a strong presence of technical, statistics-based sessions, including ones relating to methods for network meta-analysis (NMA).¹ NMA is a common and widely accepted statistical method used to compare the efficacy and safety of multiple interventions that may not have been directly compared in a randomised trial.² Standard NMAs assume that trials assessing time-to-event outcomes like progression-free survival (PFS) meet the proportional hazards (PH) assumption, which states that the hazard ratio (the ratio of the hazard rate in the treated versus the control group) is constant over time.³ If trials in a network do not meet this assumption, then the relative treatment effect estimates obtained from the NMA might be biased, and these extrapolated survival estimates might not be accurate. NMA findings are often incorporated into model-based cost-effectiveness analyses (CEAs) for health technology assessment (HTA), where relative treatment effect estimates can be key drivers of quality-adjusted life year (QALY) benefit and, thus, cost-effectiveness. Therefore, minimising any potential biases that could affect these estimates is of the upmost importance from an HTA perspective. Currently, however, there is no clear consensus on the most appropriate way to address a scenario where a trial in a network does not meet the PH assumption.⁴

Alternative Network Meta-Analysis Methods in the Presence of Non-Proportional Hazards

Technical Session 128 focused on alternative methodologies for performing NMA in the presence of non-PH (when the PH assumption is violated) for at least one trial in the network.⁴ The session gave an overview of the alternative methods (summarised in Table 1), along with a detailed, step-by-step model selection guide (Figure 1).⁴

Table 1. Alternative NMA methods in the presence of non-PH

NMA Method	Overview
1-step multivariate NMA	Parametric survival functions are fitted to the hazard functions of the interventions in each trial, and a multivariate relative treatment effect is estimated based on differences in survival parameters, which are then pooled and indirectly compared across trials.^4-6 Using multiple parameters for the relative treatment effects avoids the need for the PH assumption and provides flexibility to fit the data closely. This method is labelled as a “1-step” approach as trial-specific survival curve parameters and pooled relative treatment effects across studies are estimated simultaneously.^4-6
2-step multivariate NMA based on traditional survival distributions or fractional polynomials	This method is similar to the 1-step model except that, in the 2-step approach, parametric survival distributions are fitted for each arm of each trial in the first stage and then, in the second stage, these parameters are synthesised with a multivariate NMA model.^{4, 7}
NMA with RCS for baseline hazard	In this method, RCS are used to describe the log cumulative baseline hazard of the trial, which can be estimated more efficiently as an alternative to a Cox-based NMA of IPD.^{4, 8, 9} This method primarily focuses on constant HRs but can be extended with an interaction between treatment and the natural log of time in order to estimate time-varying HRs where the HR develops as a linear function of the natural log of time.
Restricted mean survival NMA	This is a 2-step model that first estimates the differences in the RMST between treatments in a trial based on reconstructed IPD. These estimates are then synthesised in a standard NMA model, assuming the differences in the RMST within each trial are normally distributed.^{4, 10-12} Parametric models are then used to extrapolate the survival curves.^{13, 14}

Abbreviations: HR: hazard ratio; IPD: individual patient data; NMA: network meta-analysis; PH: proportional hazards; RCS: restricted cubic splines; RMST: restricted mean survival time.

There are strengths and limitations to all four of these methods, and the most appropriate method to use depends on the available data; one can refer to Figure 1 for the step-by-step model selection guide and Table 2 for an overview of the criteria used to assess the most appropriate model for each method.⁴ All methods except the 1-step multivariate NMA are able to fit parametric survival distributions for each arm of each trial as a single step, a key strength of these methods. Goodness of fit statistics, like the deviance information criterion (DIC), can be calculated to assess the fit of the NMA model to the data for only the 1-step multivariate NMA and the NMA with RCS for baseline hazard method, another strength for these approaches.⁴ A key limitation of the restricted mean survival NMA is that it is not designed to extrapolate relative treatments or facilitate discounting for an economic evaluation, so it is not suggested to use this method for CEA. Lastly, an important overarching limitation is that there is still substantial uncertainty associated with the model selection process, and formal evaluation of this process is still needed.⁴

Figure 1. Model selection process adapted for NMA

Model selection process adapted for NMA figure

Source: Adapted from Technical Session 128: Alternative Network Meta-Analysis Methods in the Presence of Non-Proportional Hazards. ISPOR International Congress, Boston, Massachusetts, 2023.

Abbreviations: AIC: Akaike information criterion; BIC: Bayesian information criterion; DIC: deviance information criterion; HR: hazard ratio; NMA: network meta-analysis; OS: overall survival; PFS: progression-free survival; PH: proportional hazards; RCT: randomised controlled trial.

Table 2. Overview of model selection criteria for each NMA method

Model selection process	1-step multivariate NMA	2-step multivariate NMA	NMA with RCS for baseline hazard	Restricted mean survival NMA
Assessment of development of the hazard over time using trial-level diagnostics	☑	☑	☑	☑
Selection of arm-level or trial-level models using AIC and visual inspection	☒ Not applicable since study-level estimates and indirect comparisons are performed simultaneously	☑	☑	☑
Assessment of NMA model goodness of fit using DIC	☑	☒ DICs across models cannot be compared as data differ	☑	☒ DICs across models cannot be compared as data differ

Green checkmark indicates selection criterion was applicable and red ’x’ indicates the selection criterion was not applicable.
Abbreviations: AIC: Akaike information criterion; DIC: deviance information criterion; NMA: network meta-analysis; RCS: restricted cubic splines; RMST: restricted mean survival time.

Overall, these alternative NMA methods offer promising solutions in the presence of non-PH. Although multiple alternative NMA methods have been developed over the past few years, these approaches have yet to be taken up widely among researchers and statisticians.⁴ Hopefully, over time, these methods are tested more rigorously and eventually put into practice more often, as effectively addressing non-PH in NMA will help to more accurately estimate long-term survival outcomes.⁴

References

Workshop WS128: Alternative Network Meta-Analysis Methods in the Presence of Non-Proportional Hazards. ISPOR International Congress, Boston, Massachusetts, 2023.
Watt, J; Del Giovane, C. Network Meta-Analysis. Methods Mol Biol. 2022;2345:187-201.
Spruance, SL; Reid, JE; Grace, M et al. Hazard ratio in clinical trials. Antimicrob Agents Chemother. 2004;48(8):2787-2792.
Cope, S; Chan, K; Campbell, H et al. A Comparison of Alternative Network Meta-Analysis Methods in the Presence of Nonproportional Hazards: A Case Study in First-Line Advanced or Metastatic Renal Cell Carcinoma. Value Health. 2023;26(4):465-476.
Jansen, JP. Network meta-analysis of survival data with fractional polynomials. BMC Med Res Methodol. 2011;11:61.
Ouwens, MJ; Philips, Z; Jansen, JP. Network meta-analysis of parametric survival curves. Res Synth Methods. 2010;1(3-4):258-271.
Cope, S; Chan, K; Jansen, JP. Multivariate network meta-analysis of survival function parameters. Res Synth Methods. 2020;11(3):443-456.
Royston, P; Parmar, MK. Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Stat Med. 2002;21(15):2175-2197.
Freeman, SC; Carpenter, JR. Bayesian one-step IPD network meta-analysis of time-to-event data using Royston-Parmar models. Res Synth Methods. 2017;8(4):451-464.
Petit, C; Blanchard, P; Pignon, JP et al. Individual patient data network meta-analysis using either restricted mean survival time difference or hazard ratios: is there a difference? A case study on locoregionally advanced nasopharyngeal carcinomas. Syst Rev. 2019;8(1):96.
Connock, M; Armoiry, X; Tsertsvadze, A et al. Comparative survival benefit of currently licensed second or third line treatments for epidermal growth factor receptor (EGFR) and anaplastic lymphoma kinase (ALK) negative advanced or metastatic non-small cell lung cancer: a systematic review and secondary analysis of trials. BMC Cancer. 2019;19(1):392.
Niglio, SA; Jia, R; Ji, J et al. Programmed Death-1 or Programmed Death Ligand-1 Blockade in Patients with Platinum-resistant Metastatic Urothelial Cancer: A Systematic Review and Meta-analysis. Eur Urol. 2019;76(6):782-789.
Wei, Y; Royston, P; Tierney, JF et al. Meta-analysis of time-to-event outcomes from randomized trials using restricted mean survival time: application to individual participant data. Stat Med. 2015;34(21):2881-2898.
Lueza, B; Rotolo, F; Bonastre, J et al. Bias and precision of methods for estimating the difference in restricted mean survival time from an individual patient data meta-analysis. BMC Med Res Methodol. 2016;16:37.