Background

Artificial intelligence (AI) and machine learning (ML) have resulted in significant enthusiasm for their promise in healthcare. Despite this, prospective randomized controlled trials and successful clinical implementation remain limited. One clinical application of ML is mitigation of the increased risk for acute care during outpatient cancer therapy. We previously reported the results of the System for High Intensity EvaLuation During Radiation Therapy (SHIELD-RT) study (NCT04277650), which was a prospective, randomized quality improvement study demonstrating that ML based on electronic health record (EHR) data can direct supplemental clinical evaluations and reduce the rate of acute care during cancer radiotherapy with and without chemotherapy. The objective of this study is to report the workflow and operational challenges encountered during ML implementation on the SHIELD-RT study.

Results

Data extraction and manual review steps in the workflow represented significant time commitments for implementation of clinical ML on a prospective, randomized study. Barriers include limited data availability through the standard clinical workflow and commercial products, the need to aggregate data from multiple sources, and logistical challenges from altering the standard clinical workflow to deliver adaptive care.

Conclusions

The SHIELD-RT study was an early randomized controlled study which enabled assessment of barriers to clinical ML implementation, specifically those which leverage the EHR. These challenges build on a growing body of literature and may provide lessons for future healthcare ML adoption.

Trial registration

NCT04277650. Registered 20 February 2020. Retrospectively registered quality improvement study.

Objectives

Clinical artificial intelligence and machine learning (ML) face barriers related to implementation and trust. There have been few prospective opportunities to evaluate these concerns. System for High Intensity EvaLuation During Radiotherapy (NCT03775265) was a randomised controlled study demonstrating that ML accurately directed clinical evaluations to reduce acute care during cancer radiotherapy. We characterised subsequent perceptions and barriers to implementation.

Methods

An anonymous 7-question Likert-type scale survey with optional free text was administered to multidisciplinary staff focused on workflow, agreement with ML and patient experience.

Results

59/71 (83%) responded. 81% disagreed/strongly disagreed their workflow was disrupted. 67% agreed/strongly agreed patients undergoing intervention were high risk. 75% agreed/strongly agreed they would implement the ML approach routinely if the study was positive. Free-text feedback focused on patient education and ML predictions.

Conclusions

Randomised data and firsthand experience support positive reception of clinical ML. Providers highlighted future priorities, including patient counselling and workflow optimisation.

Purpose

Four-dimensional (4D) computed tomography (CT) ventilation imaging is a novel promising technique for lung functional imaging. The current standard 4D CT technique using phase-based sorting frequently results in artifacts, which may deteriorate the accuracy of ventilation imaging. The purpose of this study was to quantify the variability of 4D CT ventilation imaging due to 4D CT sorting.

Methods

4D CT image sets from nine lung cancer patients were each sorted by the phase-based method and anatomic similarity-based method, designed to reduce artifacts, with corresponding ventilation images created for each method. Artifacts in the resulting 4D CT images were quantified with the artifact score which was defined based on the difference between the normalized cross correlation for CT slices within a CT data segment and that for CT slices bordering the interface between adjacent CT data segments. The ventilation variation was quantified using voxel-based Spearman rank correlation coefficients for all lung voxels, and Dice similarity coefficients (DSC) for the spatial overlap of low-functional lung volumes. Furthermore, the correlations with matching single-photon emission CT (SPECT) ventilation images (assumed ground truth) were evaluated for three patients to investigate which sorting method provides higher physiologic accuracy.

Results

Anatomic similarity-based sorting reduced 4D CT artifacts compared to phase-based sorting (artifact score, 0.45 ± 0.14 vs 0.58 ± 0.24, p = 0.10 at peak-exhale; 0.63 ± 0.19 vs 0.71 ± 0.31, p = 0.25 at peak-inhale). The voxel-based correlation between the two ventilation images was 0.69 ± 0.26 on average, ranging from 0.03 to 0.85. The DSC was 0.71 ± 0.13 on average. Anatomic similarity-based sorting yielded significantly fewer lung voxels with paradoxical negative ventilation values than phase-based sorting (5.0 ± 2.6% vs 9.7 ± 8.4%, p = 0.05), and improved the correlation with SPECT ventilation regionally.

Conclusions

The variability of 4D CT ventilation imaging due to 4D CT sorting was moderate overall and substantial in some cases, suggesting that 4D CT artifacts are an important source of variations in 4D CT ventilation imaging. Reduction of 4D CT artifacts provided more physiologically convincing and accurate ventilation estimates. Further studies are needed to confirm this result.

Purpose: 4-dimensional computed tomography (4D-CT)-based pulmonary ventilation imaging is an emerging functional imaging modality. The purpose of this study was to investigate the physiological significance of 4D-CT ventilation imaging by comparison with pulmonary function test (PFT) measurements and single-photon emission CT (SPECT) ventilation images, which are the clinical references for global and regional lung function, respectively. Methods and Materials: In an institutional review board-approved prospective clinical trial, 4D-CT imaging and PFT and/or SPECT ventilation imaging were performed in thoracic cancer patients. Regional ventilation (V 4DCT) was calculated by deformable image registration of 4D-CT images and quantitative analysis for regional volume change. V 4DCT defect parameters were compared with the PFT measurements (forced expiratory volume in 1second (FEV1; % predicted) and FEV1/forced vital capacity (FVC; %). V 4DCT was also compared with SPECT ventilation (V SPECT) to (1) test whether V 4DCT in V SPECT defect regions is significantly lower than in nondefect regions by using the 2-tailed t test; (2) to quantify the spatial overlap between V 4DCT and V SPECT defect regions with Dice similarity coefficient (DSC); and (3) to test ventral-to-dorsal gradients by using the 2-tailed t test. Results: Of 21 patients enrolled in the study, 18 patients for whom 4D-CT and either PFT or SPECT were acquired were included in the analysis. V 4DCT defect parameters were found to have significant, moderate correlations with PFT measurements. For example, V4DCTHU defect volume increased significantly with decreasing FEV1/FVC (R=-0.65, P<.01). V 4DCT in V SPECT defect regions was significantly lower than in nondefect regions (mean V4DCTHU 0.049 vs 0.076, P<.01). The average DSCs for the spatial overlap with SPECT ventilation defect regions were only moderate (V4DCTHU0.39±0.11). Furthermore, ventral-to-dorsal gradients of V 4DCT were strong (V4DCTHU R2=0.69, P =.08), which was similar to V SPECT (R2=0.96, P<.01). Conclusions: An 18-patient study demonstrated significant correlations between 4D-CT ventilation and PFT measurements as well as SPECT ventilation, providing evidence toward the validation of 4D-CT ventilation imaging. © 2014 Elsevier Inc.

Purpose

Four-dimensional (4D) computed tomography (CT) ventilation imaging is a novel promising technique for lung functional imaging. The current standard 4D CT technique using phase-based sorting frequently results in artifacts, which may deteriorate the accuracy of ventilation imaging. The purpose of this study was to quantify the variability of 4D CT ventilation imaging due to 4D CT sorting.

Methods

4D CT image sets from nine lung cancer patients were each sorted by the phase-based method and anatomic similarity-based method, designed to reduce artifacts, with corresponding ventilation images created for each method. Artifacts in the resulting 4D CT images were quantified with the artifact score which was defined based on the difference between the normalized cross correlation for CT slices within a CT data segment and that for CT slices bordering the interface between adjacent CT data segments. The ventilation variation was quantified using voxel-based Spearman rank correlation coefficients for all lung voxels, and Dice similarity coefficients (DSC) for the spatial overlap of low-functional lung volumes. Furthermore, the correlations with matching single-photon emission CT (SPECT) ventilation images (assumed ground truth) were evaluated for three patients to investigate which sorting method provides higher physiologic accuracy.

Results

Anatomic similarity-based sorting reduced 4D CT artifacts compared to phase-based sorting (artifact score, 0.45 ± 0.14 vs 0.58 ± 0.24, p = 0.10 at peak-exhale; 0.63 ± 0.19 vs 0.71 ± 0.31, p = 0.25 at peak-inhale). The voxel-based correlation between the two ventilation images was 0.69 ± 0.26 on average, ranging from 0.03 to 0.85. The DSC was 0.71 ± 0.13 on average. Anatomic similarity-based sorting yielded significantly fewer lung voxels with paradoxical negative ventilation values than phase-based sorting (5.0 ± 2.6% vs 9.7 ± 8.4%, p = 0.05), and improved the correlation with SPECT ventilation regionally.

Conclusions

The variability of 4D CT ventilation imaging due to 4D CT sorting was moderate overall and substantial in some cases, suggesting that 4D CT artifacts are an important source of variations in 4D CT ventilation imaging. Reduction of 4D CT artifacts provided more physiologically convincing and accurate ventilation estimates. Further studies are needed to confirm this result.

The purpose of this study was to quantify postimplantation migration of percutaneously implanted cylindrical gold seeds ("seeds") and platinum endovascular embolization coils ("coils") for tumor tracking in pulmonary stereotactic ablative radiotherapy (SABR). We retrospectively analyzed the migration of markers in 32 consecutive patients with computed tomography scans postimplantation and at simulation. We implanted 147 markers (59 seeds, 88 coils) in or around 34 pulmonary tumors over 32 procedures, with one lesion implanted twice. Marker coordinates were rigidly aligned by minimizing fiducial registration error (FRE), the root mean square of the differences in marker locations for each tumor between scans. To also evaluate whether single markers were responsible for most migration, we aligned with and without the outlier causing the largest FRE increase per tumor. We applied the resultant transformation to all markers. We evaluated migration of individual markers and FRE of each group. Median scan interval was 8 days. Median individual marker migration was 1.28 mm (interquartile range [IQR] 0.78-2.63 mm). Median lesion FRE was 1.56 mm (IQR 0.92-2.95 mm). Outlier identification yielded 1.03 mm median migration (IQR 0.52-2.21 mm) and 1.97 mm median FRE (IQR 1.44-4.32 mm). Outliers caused a mean and median shift in the centroid of 1.22 and 0.80 mm (95th percentile 2.52 mm). Seeds and coils had no statistically significant difference. Univariate analysis suggested no correlation of migration with the number of markers, contact with the chest wall, or time elapsed. Marker migration between implantation and simulation is limited and unlikely to cause geometric miss during tracking.

Purpose

Patients undergoing outpatient radiotherapy (RT) or chemoradiation (CRT) frequently require acute care (emergency department evaluation or hospitalization). Machine learning (ML) may guide interventions to reduce this risk. There are limited prospective studies investigating the clinical impact of ML in health care. The objective of this study was to determine whether ML can identify high-risk patients and direct mandatory twice-weekly clinical evaluation to reduce acute care visits during treatment.

Patients and methods

During this single-institution randomized quality improvement study (ClinicalTrials.gov identifier: NCT04277650), 963 outpatient adult courses of RT and CRT started from January 7 to June 30, 2019, were evaluated by an ML algorithm. Among these, 311 courses identified by ML as high risk (> 10% risk of acute care during treatment) were randomized to standard once-weekly clinical evaluation (n = 157) or mandatory twice-weekly evaluation (n = 154). Both arms allowed additional evaluations on the basis of clinician discretion. The primary end point was the rate of acute care visits during RT. Model performance was evaluated using receiver operating characteristic area under the curve (AUC) and decile calibration plots.

Results

Twice-weekly evaluation reduced rates of acute care during treatment from 22.3% to 12.3% (difference, -10.0%; 95% CI, -18.3 to -1.6; relative risk, 0.556; 95% CI, 0.332 to 0.924; P = .02). Low-risk patients had a 2.7% acute care rate. Model discrimination was good in high- and low-risk patients undergoing standard once-weekly evaluation (AUC, 0.851).

Conclusion

In this prospective randomized study, ML accurately triaged patients undergoing RT and CRT, directing clinical management with reduced acute care rates versus standard of care. This prospective study demonstrates the potential benefit of ML in health care and offers opportunities to enhance care quality and reduce health care costs.

Background and purpose

Pre- and mid-radiotherapy FDG-PET metrics have been proposed as biomarkers of recurrence and survival in patients treated for stage III non-small cell lung cancer. We evaluated these metrics in patients treated with definitive radiation therapy (RT). We also evaluated outcomes after progression on mid-radiotherapy PET/CT.

Material and methods

Seventy-seven patients treated with RT with or without chemotherapy were included in this retrospective study. Primary tumor and involved nodes were delineated. PET metrics included metabolic tumor volume (MTV), total lesion glycolysis (TLG), and SUV_max. For mid-radiotherapy PET, both absolute value of these metrics and percentage decrease were analyzed. The influence of PET metrics on time to death, local recurrence, and regional/distant recurrence was assessed using Cox regression.

Results

91% of patients had concurrent chemotherapy. Median follow-up was 14months. None of the PET metrics were associated with overall survival. Several were positively associated with local recurrence: pre-radiotherapy MTV, and mid-radiotherapy MTV and TLG (p=0.03-0.05). Ratio of mid- to pre-treatment SUV_max was associated with regional/distant recurrence (p=0.02). 5/77 mid-radiotherapy scans showed early out-of-field progression. All of these patients died.

Conclusions

Several PET metrics were associated with risk of recurrence. Progression on mid-radiotherapy PET/CT was a poor prognostic factor.