Methods
We designed and conducted a prognostic factor study following the recommendations of the Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research34 framework and reported this in accordance with REporting recommendations for tumour MARKer prognostic studies (REMARK) guideline.35
Study context and data collection procedures
The context of this single-centre observational study was a large tertiary cancer hospital in England (UK). The hospital started integrating ePROM questionnaires into care pathways for patients with lung cancers in January 2019,36 implying that the ePROM data for the current analyses were collected as part of routine care rather than in the context of a research study. Patients were automatically enrolled in the ePROM service but could actively opt-out if they wished or could decide not to complete the ePROM survey at any time point. ePROM questionnaires included disease-specific symptom questions adapted using plain English from the Common Terminology Criteria for Adverse Events (CTCAE) version 5.0 and the EuroQoL five-dimension five-level questionnaire (EQ-5D-5L); we describe these instruments in more detail below. Enrolled patients received a text message with a web link to the ePROM questionnaires either at 17:00 on the day of a new patient appointment or 3 days before a scheduled follow-up appointment. They received immediate automated feedback on their responses on completion.36
Study sample
The study retrospectively included consecutive people with advanced NSCLC who started immunotherapy between 19 April 2019 and 1 June 2022. People were included if they were (1) aged ≥18 years or older with advanced (or stage IV) NSCLC based on pathological confirmation; (2) commencing immunotherapy drugs (atezolizumab or pembrolizumab) alone or chemoimmunotherapy using atezolizumab or pembrolizumab as an immunotherapy regimen as any line of treatment. Eligible patients were all included in the study, regardless of whether they completed ePROMs or not.
Outcomes of interest
Outcomes of interest included overall survival, time-to-progression and treatment toxicities. We defined overall survival as the length of time from initiating immunotherapy to the time of death from any cause or censored at the last day of follow-up. Time-to-progression was defined as the time from initiating immunotherapy to the time of documented disease progression and censored at the last clinical visit or the time of death from any cause. In the absence of response evaluation criteria in solid tumours data for most patients in our sample, we used a clinician-anchored approach to define the time of documented disease progression as the date of the first CT scan report mentioning progressive disease in the radiologist’s conclusion, or the date of the first clinical note stating progressive disease, when the CT scan report was not documented in the electronic patient record.37 Lastly, we defined severe treatment toxicities as the onset of any severe adverse events using the clinician-reported CTCAE v5.0.38 The CTCAE v5.0 lists relevant toxicities for people with lung cancer on systemic therapies as being absent or present; toxicities are graded based on their severity and frequency on a scale from 1 (mild) to 5 (death related to adverse events). We considered treatment toxicity severe if they had been graded 3 or higher.
Demographic and clinical covariates at baseline
Online supplemental appendix 1 shows our selection of eight covariates reflecting baseline demographic and clinical characteristics. We selected these based on prognostic and prediction models from published studies39 40 and the expertise of the clinical members of our research team (CF-F, FG and JY). Data on covariates were extracted from the electronic patient record, considering a time window from 90 days before until 14 days after the start of immunotherapy treatment.
ePROMs as prognostic factors of interest
As prognostic factors of interest, we considered four ePROMs measuring QoL and symptom burden, which we describe in more detail in online supplemental appendix 2. For ePROM scores at baseline, we considered data collected in the 6 weeks before starting immunotherapy treatment. For our early change in ePROM analyses, a change in ePROM was computed as the difference between the ePROM value at baseline and the corresponding value at the first follow-up visit during immunotherapy with the ePROM completed.
The EQ-5D-5L is a validated questionnaire to capture health-related QoL.41 42 It consists of five dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression, rating each dimension at five levels ranging from 1 (no/problems) to 5 (extreme/problems). To use the EQ-5D-5L as a prognostic factor, we calculated the utility score as per the standard value set for England, of which the values range from −0.285 (extreme/problems on all dimensions) to 1 (full health).43 The EuroQoL Visual Analogue Scale (EQ-VAS) asks people to indicate their overall health on a 0–100 hash-marked, vertical visual analogue scale, with 0 indicating the worst and 100 the best imaginable health.41
For symptom burden as a prognostic factor, we used an adapted, patient-reported version of the CTCAE v5.0 asking people in plain English to rate the presence and severity of 14 disease-specific symptoms including pain, swallowing, shortness of breath, cough, coughing up blood, tiredness, appetite loss, feeling sick, vomiting, diarrhoea, constipation, numbness, pins and needles or tingling in arms/legs/hands or feet, weakness in arms/legs/hands or feet and skin rash (see online supplemental appendix 3 for the full questionnaire). Severity grades were defined as: (1) it does not stop me from doing daily activities (coded as mild); (2) it stops me from doing daily activities (moderate); (3) as a result, I struggle to care for myself (severe). A grade of 0 was coded as ‘symptom absent’. The symptom items from the questionnaire were matched with items from the European Organisation for Research and Treatment of Cancer (EORTC) QoL questionnaire along with its lung cancer specific module (EORTC QLQ-C30/LC13) and validated in clinical practice.44 The questionnaire used to measure the presence and severity of the 14 disease-specific symptoms is presented in online supplemental appendix 3. We summed up the scores to compute the symptom burden score ranging between 0 and 42, with higher scores indicating higher symptom burden. As a sensitivity analysis, we used the number of moderate to severe symptoms, defined as the count of symptoms with a grade ≥2, reported by patients in the patient-reported version of the CTCAE v5.0. The value for this factor ranged from 0 to 14, with higher score indicating more moderate to severe symptoms.
Missing data
Missing values in routinely collected data for the purpose of prognostic factor analysis can be handled in different ways, as missingness itself may provide information.45 Therefore, we performed three different methods to handle missing data for the analyses both at baseline and first follow-up: multiple imputation (in our main analyses), as well as complete case analysis and multiple imputation plus a ‘missing’ indicator for the prognostic factors of interest.
In the complete case analyses of baseline data, we only included patients with complete information at baseline; for the first follow-up analysis, we included those with complete information at baseline and first follow-up. Given the fraction of the missing values in the variables of interest was around 50%, we performed multivariate imputation by chained equations (MICE) using the information of both baseline characteristics and outcomes to iteratively impute 40 datasets.46–48 MICE assumes that the data are missing at random, which is a less restrictive assumption than complete case analysis, which requires missing completely at random. In addition, to account for potential informative missingness, we also explored the information of missing ePROMs at baseline and the first follow-up by incorporating a missing indicator with MICE.
Descriptive analyses
We summarised baseline characteristics and both baseline and the first follow-up ePROMs as means with SDs for continuous variables with non-skewed distribution, as medians with IQR for skewed-distributed continuous variables, and as numbers with percentages for binary and categorical variables. We used Kaplan-Meier curves to describe the overall survival and time-to-progression. We compared characteristics and outcomes between baseline ePROM completers and non-completers to assess the risk of non-response bias.
Modelling procedures
We performed Cox proportional hazard regression to predict overall survival and time-to-progression, and logistic regression to predict severe treatment toxicity using ePROM data at baseline and at the first follow-up visit during immunotherapy with completed ePROMs. We conducted separate analyses for each ePROM score in online supplemental appendix 2, so there were eight analyses in total (ie, four using the start of immunotherapy and four using the first follow-up as time zero). For each ePROM analysis, we fitted three models, adding up to a total of 24 models: (1) ePROM-only model, using only the ePROM score as a prognostic factor, (2) partially adjusted model, additionally adjusted for ECOG PS and PD-L1 score and (3) fully adjusted model, adjusted for the ePROM score and all nine prognostic factors (note that smoker vs non-smoker and ex-smoker vs non-smoker were considered as two separate prognostic factors) listed in online supplemental appendix 1.
To account for regression to the mean in the longitudinal multivariable cox regression analyses, we incorporated the baseline ePROM score, as well as an interaction term between change in ePROM score and months since baseline ePROM completion at the time of first ePROM completion during follow-up.
We conducted a sensitivity analysis in patients receiving first-line immunotherapy, as patients receiving first-line immunotherapy may have a different prognosis than those receiving non-first-line immunotherapy.49
We used R (V.4.2.0) for all analyses. All p values were two-sided, with a significance level of <0.05. All estimated statistics were reported with their 95% CIs.
Sample size calculations
To estimate the minimum required sample size for our analyses, we used the ‘pmsampsize’ package in R, which follows the sample size criteria for developing multivariable prediction model recommended by Riley et al.50 Online supplemental appendix 4 shows the minimum required sample sizes for fully observed cases in all planned analyses. For the fully adjusted model of the survival outcomes, the minimum sample size required is 166 and 199 for the analyses at baseline and the first follow-up, respectively. The minimum sample size for the fully adjusted model of severe treatment toxicities is 1858 and 2229 for the analyses at baseline and the first follow-up, respectively. We considered findings for analyses hypothesis-generating if they had actual sample sizes smaller than those required.
Patient and public involvement
Patients and members of public were not involved in this study.