Skip to main content
Open Access Publications from the University of California


UCLA Previously Published Works bannerUCLA

Inter-Method Performance Study of Tumor Volumetry Assessment on Computed Tomography Test-Retest Data.

  • Author(s): Buckler, Andrew J;
  • Danagoulian, Jovanna;
  • Johnson, Kjell;
  • Peskin, Adele;
  • Gavrielides, Marios A;
  • Petrick, Nicholas;
  • Obuchowski, Nancy A;
  • Beaumont, Hubert;
  • Hadjiiski, Lubomir;
  • Jarecha, Rudresh;
  • Kuhnigk, Jan-Martin;
  • Mantri, Ninad;
  • McNitt-Gray, Michael;
  • Moltz, Jan H;
  • Nyiri, Gergely;
  • Peterson, Sam;
  • Tervé, Pierre;
  • Tietjen, Christian;
  • von Lavante, Etienne;
  • Ma, Xiaonan;
  • St Pierre, Samantha;
  • Athelogou, Maria
  • et al.

Published Web Location
No data is associated with this publication.

Rationale and objectives

Tumor volume change has potential as a biomarker for diagnosis, therapy planning, and treatment response. Precision was evaluated and compared among semiautomated lung tumor volume measurement algorithms from clinical thoracic computed tomography data sets. The results inform approaches and testing requirements for establishing conformance with the Quantitative Imaging Biomarker Alliance (QIBA) Computed Tomography Volumetry Profile.

Materials and methods

Industry and academic groups participated in a challenge study. Intra-algorithm repeatability and inter-algorithm reproducibility were estimated. Relative magnitudes of various sources of variability were estimated using a linear mixed effects model. Segmentation boundaries were compared to provide a basis on which to optimize algorithm performance for developers.


Intra-algorithm repeatability ranged from 13% (best performing) to 100% (least performing), with most algorithms demonstrating improved repeatability as the tumor size increased. Inter-algorithm reproducibility was determined in three partitions and was found to be 58% for the four best performing groups, 70% for the set of groups meeting repeatability requirements, and 84% when all groups but the least performer were included. The best performing partition performed markedly better on tumors with equivalent diameters greater than 40 mm. Larger tumors benefitted by human editing but smaller tumors did not. One-fifth to one-half of the total variability came from sources independent of the algorithms. Segmentation boundaries differed substantially, not ony in overall volume but also in detail.


Nine of the 12 participating algorithms pass precision requirements similar to what is indicated in the QIBA Profile, with the caveat that the present study was not designed to explicitly evaluate algorithm profile conformance. Change in tumor volume can be measured with confidence to within ±14% using any of these nine algorithms on tumor sizes greater than 10 mm. No partition of the algorithms was able to meet the QIBA requirements for interchangeability down to 10 mm, although the partition comprising best performing algorithms did meet this requirement for a tumor size of greater than approximately 40 mm.

Many UC-authored scholarly publications are freely available on this site because of the UC's open access policies. Let us know how this access is important for you.

Item not freely available? Link broken?
Report a problem accessing this item