Large language models (LLM)
NLP has been applied to detect real-world cancer outcomes such as metastatic progression from radiology reports, pathology reports and clinical notes.96–101 As discussed above, Gensheimer et al have incorporated NLP into clinical triage.58 NLP systems for clinical trial matching are in test deployment at large healthcare networks.102
Since the late 2010s, advanced NLP models called LLM have leveraged transformer-based architectures with massive datasets (on the order of terabytes of text) to make significant breakthroughs in language understanding. While not specifically trained on biomedical or clinical text, LLMs such as Generative Pre-trained Transformer 4 (GPT-4) are capable of encoding clinical knowledge, as evidenced by their remarkable performance on medical licensing examinations and challenge problems,103–105 as well as to highly specialised topics such as medical physics.106 Researchers are now experimenting with fine-tuning pretrained models to focus specifically on medical applications, as seen in Med-PaLM 2 for question/answering, RadOnc-GPT for oncology-specific tasks and LLMs for clinical trial matching.102 107 108
In some cases, models are even trained from scratch using solely medical datasets to optimise their performance in the healthcare context. A prime example of domain-specific NLP utilisation is the Clinical Bidirectional Encoder Representations from Transformers (ClinicalBERT) model, which was trained on a large corpus of deidentified intensive care unit notes and fine-tuned to predict short term readmission.109
The technological leaps provided by LLM have led to high-profile healthcare collaborations. Recent collaborations include between Epic Systems and Microsoft to integrate GPT-4 by drafting of patient communication responses and providing data visualisation recommendations and between Mayo Clinic and Google Cloud, which builds on a partnership formed in 2019.
Research in this area has grown rapidly with several research groups training their own BERT-type and GPT-type models on their own datasets for various applications, and we recommend keeping a close eye on this space.
3.1.2. Challenges of NLP in healthcare
Despite showing strong performance on benchmark tasks, significant risks still remain for the use of LLM in healthcare and oncology. One prominent concern is the presence of bias within these models, which may inadvertently perpetuate or exacerbate existing healthcare disparities. Additionally, LLM are known to experience ‘hallucinations,’ generating plausible sounding yet incorrect or unrelated information, which could potentially lead to detrimental clinical decisions. Furthermore, these models may still exhibit inaccurate yet plausible reasoning, thereby making it difficult to catch errors and omissions. It is crucial to address these challenges and ensure these technologies are used responsibly, with human oversight remaining integral to decision-making processes.
Digital twins (DTs)
A DT is a virtual replica of a physical system which is not only created to mirror the real-world system but is also capable of analysis and prediction. DTs continuously monitor patients in real time, integrating data from wearable devices, sensors and electronic health records and thus is complemented by other technologies, including transfer learning, the Internet of Things, edge computing and cloud computing.110 DTs are being explored in oncology as a promising approach to enhance cancer care and may be used in various aspects of oncology including drug discovery and personalised treatment planning.111
Drug discovery
DTs have demonstrated the potential to streamline pharmaceutical processes and generate realistic input–output predictions for biochemical reactions. Through in silico techniques, several drugs have been identified and successfully brought to market for various diseases, including anticancer agent raltitrexed.112 In silico trials are currently being investigated, initially focusing on synthetic control arms and eventually expanding to predict clinical intervention. Both the US FDA and the European Medicines Agency have taken steps to support the integration of in silico approaches into control arms. For instance, a synthetic control arm consisting of 68 patients was used to extend the coverage of targeted therapy for NSCLC specifically alectinib, across 20 European countries.113 Synthetic controls have also played a role in expanding the indication of palbociclib, a kinase inhibitor, to include men with HR-positive HER2-negative advanced or metastatic breast cancer, as well as facilitating accelerated approval for blinatumomab which treats acute lymphoblastic leukaemia.114 In a phase I trial, existing quantitative systems pharmacology model of the anti-CD20/CD3 T-cell engaging bispecific antibody, mosunetuzumab, were used to incorporate different dosing regimens and patient heterogeneity within the trial.115
Treatment planning and prognosis monitoring
DTs may be created to personalise treatment planning as they enable the simulation and optimisation of treatments by integrating patient-specific genomics, imaging and clinical information. The utilisation of NLP for large-scale labelling of CT reports presents an opportunity to advance the development of DTs in oncology. In a recent study, NLP was used to perform consecutive multireport prediction of metastases, enabling highly detailed representations that effectively model a cancer patient’s disease progression over time.116 These approaches facilitate the generation of a comprehensive database consisting of patterns of disease spread, facilitating early detection and prediction of an individual patient’s progression. In another study, DTs of patients were generated, and clinical trials were simulated to anticipate the optimal salvage therapy following progressive disease while on pembrolizumab.117 For spine metastases, DT was used to simulate vertebroplasty and its impact on mechanical stability of the vertebra.118 Finally, so called ‘virtual imaging trials’ aim to simulate the entire radiological imaging process using realistic digital phantoms, simulated image acquisition and reconstruction and AI-driven readers/computational observer models to improve the precision and accuracy of imaging systems and downstream biomarkers on DTs.119 Further research, validation and clinical trials are needed to fully establish the effectiveness and integration of DTs into routine clinical practice in oncology.
Clinical informatics
There are several resources that can help non-AI expert clinicians in the interpretation and ethical application of AI tools in the clinic120 121 and the applications within specific disease sites, both during and after clinical training.
Education
In the USA, the clinical subspecialty of CI was recognised by the American Board of Medical Specialties (ABMS) in 2011 and the first physicians were board certified in CI in 2013.122 CI subspecialty fellowships are open to physicians from any ABMS specialty and allow fellows to spend 2 years dedicated to studying and practicing CI. NIH National Library of Medicine informatics fellowships can also provide physicians with opportunities to gain experience in programming and application of AI tools in clinical practice. There are other pathways123 and less formal educational resources, including master’s degrees or certificate programmes, American Medical Informatics Association 10×10 programs and massive open online courses on ML such as those on Coursera by Andrew Ng.
Research infrastructure and community
Sharing of data and conference resources in oncology are increasing, even outside of data access statements in publications. Federated learning can facilitate AI from much larger datasets while protecting data privacy by decentralising raw data, which has the potential to speed up validation of models. Work continues to try to standardise oncology data elements—mCODE, a collaboration between ASCO, CancerLinQ and MITRE—and interoperability—FHIR by HL7. The National Cancer Institute (NCI) is a major organiser of cancer datasets like the Cancer Research Data Commons, which includes The Cancer Genome Atlas, and the NCI Data Catalog and NCI Cancer Imaging Archive. Academic groups are building free software packages and platforms such as MultiAssayExperiment and CURATE.AI.124 AI-driven data fusion techniques that intelligently combine data from these different source domains (eg, clinical, imaging, omics, etc) can integrate knowledge to provide insight that is greater than the sum of the parts.125 More than a dozen technology companies are building platforms and software as a service (SaaS) tools to try to facilitate precision oncology and data analysis, including ConcertAI, Onc.Ai, Azra AI, ArteraAI and PreciseDx. It is imperative that oncologists become comfortable with critiquing, interpreting and applying these tools in clinical practice as well as research.