Time trends in health-related quality of life assessment and reporting within publications of oncology randomised phase III trials: a meta-research study

Introduction

Patient-reported outcomes (PROs) should play a crucial role in determining the value of treatment for patients with cancer.1–3 The subjective nature of PROs, which are evaluated by patients based on their own perceptions, imply per se a more accurate reflection of the patient’s view about the impact of treatment on their symptoms and quality of life (QoL).4 The availability of PROs and QoL data helps clinicians to communicate the benefits and the risks of a treatment, beyond traditional investigator-assessed endpoints. Radiological control of the disease and length of survival, which represent the traditional study endpoints for treatment activity and efficacy, are not the only relevant outcomes for patients, particularly for those with advanced disease.5 6

In a previous systematic review of 446 phase III trials conducted in solid tumours and published in major oncology journals between 2012 and 2016, we found that a relevant proportion of trials did not include QoL among study endpoints.7 Namely, QoL was not included among endpoints in 47.1% of all trials, in 40.1% of trials conducted in the advanced/metastatic setting, 39.7% of industry-sponsored trials and 53.6% of academic trials. Furthermore, we found that QoL results, even when collected, were often not reported in the primary publication: out of 231 primary publications of trials with QoL as secondary or exploratory endpoint, QoL results were absent in 38.1% of cases, in 37.6% of publications in the advanced/metastatic setting, in 37.1% of industry-sponsored and 39.3% of academic trials. Disappointingly, the proportion of trials not including QoL as endpoint or not reporting QoL results was relevant in all tumour types and for all treatment categories. In addition, when analysing the timing of secondary publications including QoL results, we frequently observed a relevant delay in the publication.

In recent years, the value of PROs has been repeatedly recognised by regulatory agencies and scientific societies.8–11 The European Medicines Agency and the US Food and Drug Administration have both provided guidance for their use specifically in the setting of oncology clinical trials,8 9 and some societies incorporated the data into instruments developed to define the value of a treatment, such as the European Society for Medical Oncology -Magnitude of Clinical Benefit Scale.10 11 To allow a complete assessment of treatment value, PROs and QoL results should be published concurrently with efficacy and safety data.12 13 However, according to our previous analysis7 and other studies,14 this was often not the case, due to the above-described deficiencies in the inclusion of QoL among endpoints and in the reporting of QoL results in the publications.

With the aim of assessing time trends in QoL assessment and reporting, we reviewed phase III oncology trials published in major oncology journals between 2017 and 2021, and compared the results with the previous 5 years. In addition, we analysed the whole 10-year dataset in order to describe the characteristics of the trial and of the journal associated with the inclusion of QoL among endpoints and with the reporting of QoL results in the publication.

Methods

For this analysis, we chose the same 11 major journals that were included in the previous search to compare the results for the period 2017–2021 with the results for the period 2012–2016 considered in the previously published report.7 All the issues of these journals were screened for primary publications of randomised phase III trials testing anticancer drugs in adult patients with solid tumours (Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart in the online supplemental data). Inclusion criteria were the same as the previous analysis.7 Trials testing supportive care drugs were excluded, unless their outcome was anticancer efficacy. Trials testing non-pharmacological interventions were not included, as well as trials conducted in paediatric patients (<18 years old) and in haematological malignancies, and trials testing prevention strategies.

Data were collected using a dedicated electronic form. For each study, details about publication (journal, year, first author, date of definitive and ahead-of-print publication, availability of online supplemental material and/or study protocol) were collected. Impact factor (IF) corresponding to the year of publication was considered, according to the Journal of Citation Reports, and papers were conventionally divided into three IF categories: low (<15), intermediate (15–30) and high (>30). These cut-offs were based on the IQR of the studies included in the 2012–2016 analysis, and for homogeneity, we maintained the same cut-offs for the 2017–2021 period. Among the information recorded about the trial, those included study sponsor (industry-sponsored vs academic), type of primary tumour, disease setting (adjuvant vs neoadjuvant vs advanced/metastatic), study design (superiority vs non-inferiority), masking (open label vs blinded), details of treatment in both experimental and control arms. Trials were considered as industry sponsored when sponsored by the drug company and as academic when sponsored by an academic institution or a cooperative group, even if receiving drug supply and/or economic support from one or more drug companies. Experimental treatments were classified into four main categories (not mutually exclusive): chemotherapy±other drugs; targeted agents±other drugs; hormonal treatment±other drugs; immunotherapy±other drugs.

Endpoint information (primary/secondary/exploratory) was obtained from the paper, the study protocol (when available as online supplemental material), or the ClinicalTrials.gov record (when available). The presence of QoL details in the publication (in the main text and/or in online supplemental appendix) was recorded. For all records, secondary QoL publications were searched in PubMed, by using the name of the drug(s) and/or tumour type and/or the name of authors of the primary publication and/or the study acronym, when available. Time to secondary QoL publication was calculated according to Kaplan-Meier method, from the date of primary publication to the date of secondary QoL publication, if any, or to the date of last PubMed check (15 April 2022).

Results of the 2017–2021 period were compared with trials published in 2012–2016 for three endpoints: (1) proportion of publications including QoL among study endpoints out of all the eligible publications; (2) proportion of publications presenting QoL results out of those including QoL among endpoints and (3) proportion of publications presenting QoL results out of all the eligible publications. For the main comparison (2017–2021 vs 2012–2016) and for subgroup analysis based on study or publication characteristics (journal IF, study sponsor, type of tumour, disease setting, study design, masking, type of experimental treatment), the three endpoints were expressed in terms of OR with 95% CI. In addition, for each of the three endpoints, multivariate analysis was performed by logistic regression model in the whole 10-year dataset.

Patients and the public were not directly involved in the design and conduct of this research. However, considering the implications of our results for the discussion within scientific community, we plan to interact and cooperate with patients’ associations in oncology to disseminate and discuss our results.

All analyses were performed with IBM SPSS Statistics for Windows, V.27.0.

Results

Inclusion of QOL among study endpoints

QoL was included among study endpoints in 263 (67.8%) trials published in the period 2017–2021 (online supplemental tables 1 and 2). Namely, in these 5 years, QoL was included among endpoints in a higher proportion of industry-sponsored trials (82.3%) compared with academic trials (47.5%), with a higher difference compared with 2012–2016. Like in the previous 5 years, the inclusion of QoL among endpoints was higher in journals with higher IF (high IF 78.5%, intermediate IF 43.4% and low IF 41%) and in the subgroup of trials conducted in patients with advanced/metastatic disease (75.6%) compared with trials conducted in the (neo)adjuvant setting (50.0%).

Overall, the proportion of trials including QoL among endpoints in the period 2017–2021 was significantly higher compared with the period 2012–2016 (67.8% vs 52.9%, OR 1.87, 95% CI 1.41 to 2.48, p<0.001) (figure 1). Compared with the previous 5 years, the proportion of trials including QoL among endpoints was higher for all types of tumours, for all types of experimental treatment and in both the localised stages (OR 1.88, 95% CI 1.12 to 3.16) and advanced/metastatic setting (OR 2.07, 95% CI 1.45 to 2.95).

Request permission

Figure 1

Probability of inclusion of quality of life (QoL) among study endpoints, in the period 2017–2021 (n=388) compared with the period 2012–2016 (n=446), according to characteristics of trial and publication.

At multivariate analysis (figure 2), the inclusion of QoL among endpoints was significantly higher in journals with higher IF, in non-inferiority studies, in industry-sponsored trials, in studies conducted in patients with advanced disease, in studies conducted in genitourinary tumours and in studies testing immunotherapy.

Request permission

Figure 2

Multivariate analysis of probability of inclusion of quality of life (QoL) among endpoints and probability of presence of QoL results in the primary publication in the whole 10-year series of trials. Bold values identify the statistically significant covariates (p<0.05).

Presence of QOL results in the primary publication of trials including QOL among endpoints

Out of 263 primary publications of trials published in 2017–2021 with QoL among endpoints, QoL results were available in 137 (52.1%) (online supplemental tables 1 and 3). In these 5 years, the proportion of publications without QoL results was 20.0%, 47.8% and 51.2% among papers published in journals with low, intermediate and high IF, respectively. The proportion of publications without QoL results was 51.6% among industry-sponsored trials and 39.0% among academic trials, and it was 47.5% in localised stages and 48.0% in the advanced setting.

The overall proportion of trials presenting QoL results in the primary publication in the 2017–2021 period was significantly lower compared with the period 2012–2016 (52.1% vs 62.3%, OR 0.66, 95% CI 0.46 to 0.94, p=0.02) (figure 3A). Compared with 2012–2016, the proportion of publications reporting QoL results was lower in many types of tumours, for all types of treatment and in both the localised stages and advanced/metastatic setting.

Request permission

Figure 3

Probability of presence of quality of life (QoL) results in the primary publication, in the period 2017–2021 compared with the period 2012–2016, according to characteristics of trial and publication. (A) Trials including QoL among study endpoints (n=263 for the period 2017–2021 vs n=236 for the period 2012–2016). (B) All trials included in the analysis (n=388 for the period 2017–2021 vs n=446 for the period 2012–2016).

At multivariate analysis, higher journal IF and studies testing immunotherapy were the only variables associated with a lower probability of QoL results (figure 2).

Qol secondary publications

Overall, out of 126 trials published between 2017 and 2021, including QoL as an endpoint but without any QoL result in the primary publication, 42 secondary QoL publications were found with a median follow-up of 34 months, (the complete list of secondary publications is available in online supplemental data). For these trials, probability of secondary publication was 10.9%, 29.1% and 42.5% at 1, 2 and 3 years respectively (figure 4). These results were similar to those of papers published between 2012 and 2016, included in the previous analysis.

Request permission

Figure 4

Kaplan-Meier curves of time to secondary publication with quality of life (QoL) results, for trials including QoL as a secondary/exploratory endpoint, but without any QoL result in the primary publication.

Discussion

In this study, we showed that the proportion of randomised phase III trials that included QoL as a study endpoint has increased in recent years, though it remains suboptimal, particularly in trials promoted by academic sponsors.

In many cases, although collected according to study protocol, QoL data are not presented in the primary publication. This under-reporting was already relevant some years ago,7 but the trend is worsening in recent years, with a frequent delay in the availability of QoL results. For instance, even if the rate of industry-sponsored trials including QoL among endpoints has increased in the last 5 years, this was not followed by an increase in reporting of QoL results, which on the contrary has decreased in 2017–2021 compared with the previous 5 years. The same concept also applies to trials testing immunotherapy or conducted in advanced disease, among others. It should be highlighted that the drop in the presence of QoL results in primary publications was particularly relevant for trials including patients with lung cancer (45.8% in 2012–2016 compared with 25.0% in 2017–2021). Of course, regulatory agencies may evaluate QoL results although still unpublished, but we believe that the concomitant publication of QoL results together with other outcomes is very useful not only for decision makers, but for the entire scientific community (including clinicians and patients) to evaluate the value of a new treatment.

By performing a multivariate analysis on the whole 10-year dataset, we investigated the factors (both related to the study and journal characteristics) associated with a higher chance of inclusion of QoL among study endpoints and presence of QoL results in the primary publication. Of note, journal IF was associated with the inclusion of QoL among endpoints and with the presence of QoL results in the first publication. However, these associations were in opposite directions: studies published in journals with higher IF had higher frequency of QoL among endpoints (multivariate OR for journals with high IF compared with those with low IF 4.13, 95% CI 2.59 to 6.56), but higher IF was associated with lower inclusion of QoL results in the first publication (multivariate OR 0.37, 95% CI 0.19 to 0.75). In other words, if we consider the journal IF as a surrogate of the quality and the relevance of the trial, high-quality trials do include QoL in study protocol, but those journals do not ask for those results when the study is submitted for publication. This is particularly evident for trials testing immunotherapy, as previously shown by a dedicated analysis.15 In this study, immunotherapy studies confirmed a high rate of inclusion of QoL among endpoints, but a very disappointing rate of presence of QoL data in the primary publication.

As expected, the proportion of trials including QoL among endpoints was significantly lower in trials conducted in early stages compared with advanced disease, although in the last 5 years this proportion showed an improvement in both subgroups. Disease setting showed a significant association at multivariate analysis, with both inclusion of QoL among endpoints and presence of QoL data in the primary publication. The attention to QoL was higher in patients with advanced disease: indeed, in this setting, tumour-related symptoms can have a relevant impact on QoL, and the burden of treatment-related adverse events is not negligible, especially if treatment is associated with a modest efficacy. In patients receiving adjuvant treatment, with a curative aim, investigators often suppose that the benefit in terms of chance of cure should outweigh the negative, hopefully transient, detrimental effect.

We were already disappointed in the previous analysis by the high proportion of academic trials that did not include QoL as a study endpoint. In this update, the gap between industry-sponsored and academic studies was even higher: while the proportion of trials including QoL among industry-sponsored studies significantly improved in recent years (moving from 60.3% to 82.3%, OR 3.06), this was not the case for the academic counterpart (from 46.4% in 2012–2016 to 47.5% in 2017–2021, OR 1.05). In the multivariate analysis conducted over the 10-year period, study sponsor remained statistically significant (OR of QoL inclusion for industry-sponsored vs academic trials 1.48, 95% CI 1.01 to 2.17). In other words, pharmaceutical companies are responding to regulatory agencies’ invitation to increase the production of QoL data during a treatment’s clinical development, while academic research continues to underestimate the importance of PROs and QoL in clinical trials. We acknowledge that this can be related to technical and methodological challenges: academic trials are often conducted with far less resources compared with industry-sponsored trials, and the administration, collection and analysis of QoL questionnaires could be considered as a ‘burden’ mining the feasibility of the trial. However, we hope that the increased awareness of the relevance of QoL will determine an improvement also in academic research, coherently with the its philosophy, aiming to optimise treatment choices and patients’ QoL.

One reason for not including QoL as an endpoint could be the concern that, in order to be methodologically reliable, QoL data should be generated in a blinded setting rather than in an open-label design.9 If this was actually the case, this could partially explain the suboptimal QoL inclusion in academic research, considering that the conduction of placebo-controlled trials can be particularly challenging for academic investigators. In our analysis, masking was significantly associated with the probability of QoL results in the study publication at multivariate analysis (OR 1.52, 95% CI 1.07 to 2.18). However, we believe that QoL data are worth to be collected even if the trial is open-label. In a review of 538 randomised trials with a patient-reported endpoint conducted in the most prevalent cancers, there was no significant association of the treatment concealment (blinded vs open-label) with the proportion of trials favouring the experimental treatment (adjusted OR 1.19, 95% CI 0.79 to 1.79; p=0.40).16 These findings support the validity of QoL results derived from open-label oncology trials.

The limited word-count imposed by most scientific journals may be one of the factors hindering QoL reporting in the primary publication. Limiting the reporting of QoL results to the global/total QoL score due to space restrictions may limit the interpretation of the QoL profile of the treatments under study. In fact, results of the various QoL items and domains, along with different modalities of analysis and presentation of results (eg, mean changes, proportion of responders, time to QoL deterioration), are important to understand treatment impact on QoL and to personalise treatment choices and management. We believe that scientific journals should consider synchronous publications dedicated to PROs, in order to offer a timely and complete evaluation of new treatments. Alternatively, authors should be encouraged to add details of PROs results in the supplemental data, which usually have no word-count restriction, although this solution is at risk of limiting the visibility of PRO results.

We acknowledge some limitations of our study. First, our analysis is limited to 11 major journals and cannot be considered a systematic review of the literature. Some phase III trials published in different journals, even of high relevance and quality, are not considered in our study. However, considering our finding that studies including QoL among endpoints are published more often in journals with higher IF, it is reasonable that the inclusion of other journals (including those with lower IF) could have further reduced the proportion of studies including QoL; therefore, this limitation is conservative as for the matter of the analysis. In addition, we decided to conduct the update on the same 11 journals already included in our previous study, to obtain a homogeneous comparison between the two periods, and the journals included are among those most read by the oncological community. We believe that, despite this important limitation, our review is useful in providing a picture of the relevance of QoL in oncology literature. Second, this analysis, like the previous one, is limited to full text publications. We did not consider the presentation of QoL results as abstracts, posters and/or oral presentations, at scientific meetings, although those usually precedes the definitive publication and could allow the availability of QoL data for the scientific community for some of the trials classified as under-reported in our analysis. However, we believe that the full text, peer-reviewed publication should be the reference for the evaluation of the methodological quality of a clinical trial. In addition, there is no scientific reason for not including QoL data in the primary study publication, considering that those data are collected during the treatment, and should be mature at the moment of primary analysis. Third, we did not investigate the modality of collection (paper-based vs electronic) within each study. Recent years are witnessing an increasing use of electronic collection of PROs, which allows a prompt availability of data for the analysis. Whether the use of electronic PROs may be associated with higher PROs reporting still needs further investigation. Lastly, we acknowledge that the year of publication is an imprecise surrogate for the year of study design and that the length of accrual for academic trials could be, at least in principle, longer than for industry-sponsored trials due usually to more limited resources. This can affect the comparison of time trends according to sponsorship, at least for the inclusion of QoL among study endpoints, but not for the presentation of results.

In conclusion, we showed that the proportion of oncology trials including QoL among endpoints increased in 2017–2021 compared with 2012–2016, and this is an encouraging result, although QoL remains underrated in many trials. Furthermore, the proportion of trials reporting QoL results in primary publications remains suboptimal, necessitating discussion within the scientific community, which includes not only investigators but also journal editors and peer-reviewers. We believe that more needs to be done to ensure that the scientific community and also patients can access data of PROs and QoL at an appropriate time. At the moment, journal editors, who could make the provision of PROs and QoL for clinical trials a requirement for publication, may hold the key for the improvement of the timely presentation of patient-reported results.