Feasibility of implementing a rapid-learning methodology to inform radiotherapy treatments: key professional stakeholders’ views

Study aim

In this article, we report on the findings from a study that aimed to explore the feasibility and acceptability of implementing rapid-learning in the clinic. The study examines key professional stakeholders’ views towards the potential strengths and challenges of implementing rapid-learning in practice.

Design and setting

This study adopted a multicentre, qualitative interview-based design. Five UK cancer centres participated in this study: The Christie National Health Service (NHS) Foundation Trust (Manchester) (host study site), The Royal Marsden NHS Foundation Trust (London), Leeds Cancer Centre (St James’s Institute of Oncology, Leeds Teaching Hospitals NHS Trust, Leeds), Worcestershire Oncology Centre (Worcestershire Acute Hospitals NHS Trust, Worcester) and James Cook Cancer Institute (South Tees Hospitals NHS Foundation Trust, Middlesbrough). These sites were chosen as (a) they were geographically diverse, (b) they reflected different healthcare environments (varying in size and academic capacity) and (c) they served populations with varying demographic characteristics. Data collection took place between December 2022 and May 2023.

Participants and procedure

Participants were eligible for the study if they were a professional based across the following areas: (1) clinical oncology, (2) radiotherapy physics and treatment planning, (3) digital learning, informatics and innovation, and (4) research and information governance. Gathering these different perspectives was important as introducing a change in radiotherapy requires cross-department collaboration. Purposive sampling was used with participants identified through consulting with clinical service leads at respective sites. After identification, individual study invites and participant information sheets (PIS) were circulated via email to participants by the RAPID-RT study team (AK) asking whether they wished to participate in a one-off interview.

In total, 23 participants were interviewed (table 1). For several reasons, a large number of these interviews took place with personnel based at The Christie (Manchester). This is the first site at which implementation of rapid-learning in lung cancer radiotherapy takes place and we further anticipated that professionals may be, to some extent, more familiar or knowledgeable about rapid-learning. Subsequently, this sample represented ‘information-rich’ cases with informed views that we intended to capture.33 All participants were knowledgeable about the types of data that may be grouped under RWD. Knowledge of the rapid-learning concept and methodology varied; knowledge appeared greatest among clinical professionals (clinical oncologists, medical physicists and treatment planning) based in the large cancer centres. None of the participants interviewed were members of the immediate RAPID-RT study team. Data collection continued until the research team were satisfied that sufficient data had been collected and no new information was being generated. In this manner, our study sample size was informed by the concept of information power34—the more relevant information a sample holds, the fewer participants are needed.

Interviews and topic guide

Semi-structured interviews were conducted virtually (over MS teams or Zoom) and ranged between 35 and 60 min; the median interview time was 45 min. Semi-structured interviews were considered an effective method to explore participants’ perceptions of rapid-learning while encouraging further questions to emerge from the discussion itself.

Two original interview guides were developed between members of the RAPID-RT research team comprising a range of expertise that includes qualitative methods, implementation science, health psychology, radiotherapy and oncology. While sharing several questions, the first guide (designed for clinic-based professionals) involved a slightly greater focus on understanding clinical evidence, whereas the second guide (designed for management, data informatics and research personnel) entailed more focus on data and governance-related areas. Key areas such as data handling, governance and infrastructure, patient facing workflows and operational capacity were covered across interview guides (online supplemental file 1). For purposes of making abstract ideas more concrete for participants less familiar with the ideas being discussed, a clinical exemplar was attached to the PIS as a pre-interview reading exercise for all participants (figure 1). In the interview, this RAPID-RT clinical exemplar was introduced in more detail by the interviewer (AK) to help stimulate further discussion. The timing of introducing the exemplar during interviews differed based on familiarity shown by participants towards rapid-learning.

Analysis

The majority of interviews were audiorecorded and transcribed verbatim by an independent professional transcription team. One participant did not consent for their interview to be recorded and notes were taken instead. Data were analysed in NVivo V.12 using thematic analysis.35 Thematic analysis enables the identification of representative patterns of participants’ views and experiences from across the dataset. Each transcript was systematically read multiple times for familiarisation prior to coding by AK. Coding was conducted at an inductive-manifest level, whereby codes were derived from the data rather than being guided by preconceived ideas. Coding was iterative with developing codes compared and refined across transcripts. Patterns were identified within the codes and initial themes were created by AK. Themes and codes were compared across the dataset with analysis concluded when no new codes were generated from the data. Codes, developing themes and the final thematic structure were reviewed and refined by AK, DPF, GP and CF-F (online supplemental file 2).

Theme 1: the alignment of rapid-learning methodology with the reality of practice

This first theme captures participants’ views on the current evidence landscape, and how rapid-learning may positively address evidence gaps and complement practice. RCT evidence was described as gold standard and robust, with participants noting high internal validity (ability to determine cause-and-effect relationships) as a particular strength. This said, overwhelmingly, all participants discussed RCTs in response to the challenges experienced in practice. Along with time, finance and resource costs of preparing and conducting RCTs, participants highlighted strict eligibility criteria, often applied in trial recruitment and resulting in the participation of certain patient groups (‘self-selected,’ ‘age-specific’ patients), as a barrier to generalising trial outcomes to the real-world population.

To address some of the identified challenges of RCTs, participants reflected on how rapid-learning could provide a complementary methodology. Participants, for example, viewed access to routinely collected RWD and the cyclical design of rapid-learning as an opportunity to generate evidence within a more reasonable time frame than currently experienced. Moreover, clinical oncologists and treatment planning personnel discussed how rapid-learning and RWD could offer potential to provide more personalised radiotherapy treatment; this was by virtue of RWD containing information about diverse patient communities. Personalisation of treatment was highlighted as an ongoing development within radiotherapy and represented a particular ambition for all centres. Participants described how personalisation not only offered potential for greater treatment choice but could also help optimise resources and minimise wastage.

Clinic-based professionals, across centres, recalled various experiences of making incremental changes to radiotherapy techniques and practices that were not informed by ‘highest quality’ RCT evidence. Situations where such incremental changes were introduced included changing dose limits and applying organ-at-risk constraints, responding to uncommon tumours and diagnoses, and delivering radiotherapy through uncertain service periods, for example, the COVID-19 pandemic. Exploring the rationale behind introducing incremental changes to techniques, participants highlighted several factors that included clinical protocols and guidelines, clinician experience, local consensus among teams and auditing practices. Clinical audit appeared to be customary practice across all centres. Participants, who were less familiar with rapid-learning, likened the cyclical approach in rapid-learning to their clinical audit practices on review of the clinical exemplar; the use of RWD to evaluate outcomes represented the major difference. Subsequently, rapid-learning was viewed not too dissimilar from existing practices currently employed and potentially offered a way to ‘formalise’ processes already taking place such as introducing incremental changes.

Theme 2: concerns related to the variability of RWD

This second theme captures participants’ views regarding the quality of RWD and how this could present a challenge to rapid-learning implementation. All participants raised some scepticism over the accuracy and robustness of clinician-reported and patient-reported datasets as they drew on their respective experiences. Participants described how datasets can often be incomplete and missing data, while data may not always be recorded systematically (eg, use of different data recording formats) making it difficult to analyse. The collection of clinician-reported data, for example, was described on several occasions by participants as a burden with limited time and resource for healthcare staff. In discussion around toxicity data, vital to evaluating outcomes in rapid-learning, the informativeness of this data could be impacted by the failure to collect robust follow-up data. This concern was expressed by staff across all centres which they placed on losing contact with individuals after their discharge.

The completeness of patient-reported outcome measure (PROM) data was also raised. Some doubts were cast over the accuracy of PROM data, with some clinicians noting how, in their view, this type of data was not always well documented, for example, under-reporting or over-reporting of symptoms. Equally, issues regarding accessibility (eg, ability for certain groups to provide this data and IT literacy) were raised regarding the collection of PROM data. Though considered vital for measurement of outcomes deemed important by patients, the extent to which PROM data were collected and its quality varied between centres. Professionals based in Worcester (small non-academic centre), Middlesbrough (medium-sized acute trust) and Leeds (large academic centre) were conscious that they did not collect PROM data post-treatment (although this did form an ongoing programme of work at Leeds). In comparison, the collection of PROM data at The Christie (large academic centre) appeared well established. Meanwhile, at the time of this study, a new electronic patient record platform called Connect was being readied for launch at Royal Marsden (large academic centre) with an aim to improve patient engagement that included the ability to interact with clinicians and contribute to care records.

From participant discussion, it appeared that larger centres viewed the analysis of data already collected as a key challenge, while both collection and analysis of key data presented challenges for the smaller and medium centres participating in the study. Nonetheless, as RWD encompasses multiple varieties of data and is key to evaluate outcomes in the rapid-learning approach, all participants were conscious of how data quality may affect the robustness of rapid-learning in practice.

Theme 3: the maturity of data infrastructures and capacity for rapid-learning

This third theme describes the role of data infrastructures to address identified challenges regarding data quality and the capacity of centres to implement rapid-learning. Participants based in large centres roundly stressed the importance of integrating clinical and digital services to address data handling challenges and improve data interoperability. Digital learning and treatment planning personnel expressed the importance of greater data standardisation and accessibility, which they felt could be achieved through developing informatic infrastructures. Participants recognised that this would require time, investment and organisational commitment. Further potential benefits of more mature data systems, as highlighted by participants, included the collection of more granular detail and achieving greater data linkage and sharing between departments and within organisational boundaries. The following mantra—‘availability of the right data at the right time’—was repeated on several occasions by participants in response to often negative experiences of using complex data platforms.

Reflecting on the development needed to address data quality challenges, significant consideration was given by participants over the capacity of different cancer centres to implement rapid-learning. Identified set-up differences between cancer centres that participants believed could impact implementation included staff turnover, varying clinical interest and experience, difference in quality control mechanisms and capacity to build local evidence bases. Professionals based in Worcester (small non-academic centre) and Middlesbrough (medium-sized acute trust) respectively expressed some caution over their ability to implement rapid-learning citing lack of digital infrastructure, research time and space, and resources (therefore unable to build local evidence bases), which, in their view, were more commonly visible within larger academic centres. This said professionals based in larger cancer centres described how introducing change was not necessarily straightforward in their centres either, citing factors such as complexity of departments, established methods of practice and high patient turnover.

Where difficult to build a local evidence base (as expressed by professionals from Worcester and Middlesbrough), it was expected that some steer from organisations such as The Royal College of Radiologists or similar national-level guidance would be required to inform and support plans for implementation. In this respect, participants felt it would be more realistic for smaller and medium cancer centres to be part of wider organised implementation efforts. The need for greater data linkage and sharing between organisations was further emphasised here given the variable capacity to collect and analyse data in certain cancer centres. While it was acknowledged that individual centres face respective challenges, a consistent message across all interviews was that it was important to avoid rapid-learning becoming an ‘ivory-tower concept’ that restricted development to the host site (The Christie) and similarly well-connected, large centres.

Theme 4: further support, education and evidence needed to convince stakeholders to adopt rapid-learning approaches

This fourth theme describes participants’ views towards what they considered may be needed to inform further acceptability of rapid-learning implementation across radiotherapy teams. A key methodological area that clinical professionals believed needed further clarity concerned the timeliness of learning cycles in rapid-learning. Progressing learning between rapid-learning cycles (and finding the optimal dose limit to the base of the heart in the RAPID-RT study) is contingent on measurement of acceptable toxicity.18 However, clinical professionals noted how toxicity may take time to develop, thereby potentially impacting the length of learning cycles. Subsequently, they questioned to what extent rapid-learning matched its moniker of ‘rapid’—‘how rapid is ‘rapid’?’ Furthermore, clinical oncologists described how cardiac risk is compounded by numerous factors36 37 but these data may not be routinely available or collected.38 39 Therefore, questions were raised over whether additional outcomes and measures would need to be collected as to provide more accurate toxicity data, and also how this would then impact existing routines of work. Further clarification of safety procedures in rapid-learning was also considered essential to secure ‘buy-in’ from professionals. This included, for example, clear information regarding critical data points and toxicity biomarkers, along with the processes through which trade-offs and unintended consequences would be managed. It was suggested that this could help convince radiotherapy teams of the ‘soundness’ of rapid-learning and increase confidence to link changes implemented with resultant outcomes observed.

To further address potential implementation challenges, participants noted how it would be advantageous to have dedicated channels of analytical support, particularly for data-related issues, along with more time and space for research exploration. Education for teams (particularly non-research staff) was widely acknowledged as rapid-learning is not based on standard competence and would require time and planning for familiarisation. In terms of the evidence required to change practice, proof of publication and peer review were considered minimum evidence (a practical demonstration of benefits) supported by professional debate (eg, through societies like The British Thoracic Oncology Group) with information and education dispersed nationally to different centres.

Implications for research and practice

On review of the study findings, we suggest the following implications for research and practice that may warrant further exploration. The need to collect additional baseline data or different toxicity measures may be important given that clinical professionals discussed how numerous factors impact cardiac toxicity but this data may not always be routinely collected. In the absence of such data, professionals may find it difficult to definitively attribute the changes introduced to the resultant outcomes observed. Similarly, then, study teams may need to clarify key clinical markers or conduct further research as to satisfy ethical and safety concerns, clear uncertainties and increase confidence in rapid-learning approaches.44–46

There was shared consensus among centres regarding the significant role of organisations in fostering learning cultures and creating appropriate conditions for implementation, for example, time, resource, funding, education and training.43 47–49 We suggest further exploration may be needed to review how rapid-learning implementation impacts staff workload and routines, for example, collection, reporting and analysis of data. Radiotherapy teams already encounter tension between immediate clinical service delivery and research time.50 51 Furthermore, extensive research reports strong correlation between clinician burn-out and workload demands associated with the utilisation and management of EHR data.43 52

This study will be proceeded by the next stage of the research which seeks to assess the practical experience of implementing rapid-learning in lung cancer (at The Christie).30 Interviews with radiotherapy professionals will be held towards the end of the first learning cycle with a focus on capturing experience of implementation and whether rapid-learning is meeting its initial expectations.

Strengths and limitations

Key strengths of this research study are that it included the views and experiences of a diverse range of professionals based across different cancer centres which provided greater understanding of the distinct roles and processes involved when introducing a change to radiotherapy practice. The overwhelming majority of participants were knowledgeable about current processes involved in radiotherapy practice evaluation, the various forms of evidence informing practice and RWD. Subsequently, participants were able to use this knowledge to reflect on and compare with the rapid-learning approach.

Limitations include that more interviews took place with personnel based at The Christie compared with other centres. We acknowledge that there is risk for potential bias within the findings presented as views expressed by participants may be overwhelmingly representative of a singular centre. We have explained the reasons for conducting more interviews at The Christie. The is the first site at which implementation of rapid-learning in lung cancer radiotherapy takes place and we further anticipated that some participants may be more familiar with the rapid-learning concept, thereby holding informed views that we intended to capture. It is further possible that some participants interviewed, across centres, may hold prior interest in the topic area, and therefore, agreed to participate in the study. Given rapid-learning had not been implemented at any of the sites at the time of this study, participants’ responses could be best categorised as hypothetical rather than based in practical experience. As a result, we stress that the findings presented in this article are participants’ views and the authors’ interpretation of them.