Eur Radiol (2010) 20: 1651–1656 DOI 10.1007/s00330-009-1704-z

Perry J. Pickhardt Steven M. Wise David H. Kim

Received: 11 September 2009 Accepted: 12 November 2009 Published online: 13 January 2010 # European Society of Radiology 2009

P. J. Pickhardt (*) . S. M. Wise . D. H. Kim Department of Radiology, University of Wisconsin School of Medicine and Public Health, E3/311 Clinical Science Center, 600 Highland Ave., Madison, WI, 53792-3252, USA e-mail: [email protected] Tel.: +1-608-2639028 Fax: +1-608-2630140

GASTRO INTESTINAL

Positive predictive value for polyps detected at screening CT colonography

Abstract Purpose: To determine the positive predictive value (PPV) for polyps detected at CT colonography (CTC). Methods: Assessment of 739 colorectal lesions ≥6 mm detected prospectively at CTC screening in 479 patients was performed. By-polyp PPV was analyzed according to small (6–9 mm) versus large (≥10 mm) size; morphology (sessile/pedunculated/ flat); diagnostic confidence level (3 = most confident, 1 = least confident); and histology. By-patient PPV was analyzed at various polyp size thresholds. Results: By-polyp PPV for CTC-detected lesions ≥6 mm, 6–9 mm, and ≥10 mm was 91.6% (677/739), 90.1% (410/451), and 92.7% (267/288), respectively (p=0.4). By-polyp PPV according to sessile, pedunculated, flat, and masslike morphology was 92.5% (441/477), 96.5% (139/144), 77.7% (73/94), and 97.6% (40/41), respectively (p<0.0001 for flat versus

Introduction CT colonography (CTC) has been shown in recent clinical validation and comparison trials to be comparable to optical colonoscopy (OC) for the screen detection of advanced neoplasia [1–3]. Broad consensus among radiologists on the general application of CTC as a clinical tool has been achieved [4, 5]. To function as a clinically efficacious and cost-effective screening tool, CTC must not only be sensitive for lesion detection, but also demonstrate a reasonably low false-positive rate [6–8]. In contrast to validation trials where OC is performed in all subjects [1, 3], including those

polypoid morphology). By-polyp PPV according to diagnostic confidence level was 94.7% (554/585) for highest (= level 3), 83.5% (106/127) for intermediate (= level 2), and 63.0% (17/27) for lowest (= level 1) confidence (p<0.0001 for levels-2/3 versus level-1). By-patient PPV at 6-mm, 8-mm, 10-mm, and 30-mm polyp size thresholds was 92.3% (442/479), 93.0% (306/329), 93.1% (228/245), and 97.4% (38/39), respectively. Conclusion: The overall per-polyp and per-patient PPV for lesions ≥6 mm was 92% for CTC screening. Increased diagnostic confidence and polypoid (non-flat) morphology correlated with a higher PPV, whereas small versus large polyp size had very little effect. Keywords CT colonography . Virtual colonoscopy . Colorectal cancer screening . Polyps

with negative CTC results, the sensitivity and specificity of CTC (relative to OC) cannot be measured in clinical practice. The reason for this is that negative CTC cases are not referred to OC and thus cannot be stratified into truenegative versus false-negative results. However, because both true-positive and false-positive CTC results can be tracked in clinical practice, the positive predictive value (PPV) of CTC screening can be obtained and will likely be an important quality assurance measure going forward. The purpose of this study was to determine the PPV of our CTC screening program for the detection of relevant colorectal lesions (≥6 mm), and to assess the

1652

influence of lesion size, morphology, and diagnostic confidence on the PPV.

Material and methods This Health Insurance Portability and Accountability Act (HIPAA)-compliant study was approved by our institutional review board (IRB); the need for signed informed consent was waived due to the retrospective nature of assessment. A total of 5,124 consecutive ambulatory adults (2,792 women and 2,332 men) underwent CTC screening at a single medical center over a 52-month interval. The mean age of this screening population was 56.9 years. Bowel preparation the evening before CTC screening consisted of a saline laxative (2 doses of 296 ml magnesium citrate or 1–2 doses of 45 ml sodium phosphate) and oral contrast agents (250 ml 2% w/v barium sulfate and 60 ml diatrizoate [Gastroview, Covidien]). Bowel distention for CTC was achieved with automated carbon dioxide delivery (PROTOCO2L, Bracco Diagnostics), except for an early subset of 419 patients who underwent room air insufflation. Supine and prone acquisitions were obtained with 8- or 16-channel multi-detector CT scanners (LightSpeed Series, GE Healthcare), with 1.25-mm collimation, kVp of 120, 50–75 mAs or modulated tube current with noise index set at 50 and mA range of 30–300 mA, and 1-mm reconstruction interval. For cases with inadequate segmental distention in the same location on both supine and prone positioning, an additional decubitus view was also obtained—typically right lateral decubitus position to optimize sigmoid distention. Prospective CTC study interpretation for polyp detection was performed using a dedicated workstation (V3D Colon, Viatronix). The studies were read by a one of five abdominal radiologists, all with extensive experience in CTC interpretation (>500 cases). A hybrid 3D and 2D polyp detection approach was used, with 2D correlation to confirm soft tissue composition of all suspicious lesions. Lesion measurement was performed using combined 2D and 3D assessment in a manner that has been previously described [9]. For all polyps measuring ≥6 mm, the lesion size, morphology, segmental location, and diagnostic confidence level were prospectively recorded. Polyp size was categorized as small for 6–9 mm lesions and large for all lesions measuring 10 mm or greater. Lesion morphology was characterized as sessile, pedunculated, flat, or mass-like. Pedunculated lesions had a definable stalk of attachment separate from the polyp head. Flat lesions are a subset of sessile polyps that have a plaque-like morphology and generally measure 3 mm or less in height (and always measure less than half as tall as wide) [10, 11]. Masses were defined as ≥3 cm and were subclassified as carpet lesions when relatively flat to distinguish them from polyps, annular, or semi-annular

masses. Polyp location was scored according to the six segments: cecum, ascending, transverse, descending, sigmoid, and rectum. Diagnostic confidence was prospectively assigned by the interpreting radiologist using a previously validated three-point scale [12], ranging from highest confidence (= level 3) to lowest confidence (= level 1), with level-2 confidence being intermediate. Diagnostic confidence scoring is somewhat subjective. However, a typical level-3 lesion would be composed of homogeneous soft tissue attenuation and its site of attachment to the bowel wall would remain fixed in position between supine and prone. In comparison, a typical level-1 confidence finding might be fairly subtle, associated with suboptimal preparation or distention, and perhaps clearly identified on only one view. According to our diagnostic algorithm, all patients with polyps ≥6 mm detected at CTC screening were offered same-day OC for polypectomy, unless contraindicated by a patient factor such as anticoagulation therapy. In addition, patients with one or two small (6–9 mm) polyps were given an alternative option of short-term CTC surveillance. The inclusion criterion for this study consisted of all CTCdetected lesions ≥6 mm that underwent subsequent OC evaluation. Lesion measurement at OC was performed using visual comparison against an open forceps, calibrated guidewire, or other endoscopically inserted device. For lesion matching at CTC and OC, a standard algorithm was applied that required concordance for both lesion size (within 50%) and location (same or adjacent segment) [3]. For most patients, polyp matching was very straightforward. For any case where the matching was difficult or ambiguous (e.g., multiple lesions of varying sizes), an additional consensus review was performed for reconciliation, consisting of two or more experienced CTC radiologists. Any fixed and focal mucosal-based anatomic abnormality called at OC was considered a potential candidate for matching with CTC (assuming appropriate size and location), regardless of end histology (e.g., adenomatous, hyperplastic, normal mucosa, juvenile, inflammatory, lymphoid, etc.). Some matching lesions such as venous blebs, were not biopsied, whereas some other matching lesions were fulgurated or could not be retrieved. Adherent stool was always considered a falsepositive result. A matched lesion (CTC–OC concordance) was considered a CTC true positive, whereas a CTC false positive lacked a matching correlate at OC. The CTC positive predictive value (PPV) was defined as = (CTC TP)/(CTC TP + CTC FP). True-positive versus false-positive matches were assessed by-polyp in terms of lesion size, morphology, diagnostic confidence, and histology. PPV on a per-patient basis required at least one matching lesion at OC for a patient to be considered a true positive. Per-patient PPV was assessed at 6-mm, 8-mm, 10-mm, and 30-mm polyp size thresholds. The false-positive rate (FPR) was defined as 1−PPV.

1653

Statistical analysis Fisher’s exact test was used to assess for statistical significance between predictive values according to the categories of lesion size, morphology, and diagnostic confidence. A p value <0.05 was considered to be statistically significant.

Results From the entire screening cohort of 5,124 subjects undergoing CTC, a total of 639 patients had at least one lesion ≥6 mm, corresponding to a test positive rate of 12.5%. Of these 639 patients with 958 total lesions ≥6 mm, 160 patients with 219 polyps undergoing CTC surveillance were excluded from further analysis, leaving a study cohort of 479 patients with 739 lesions ≥6 mm that were evaluated by OC. The PPV for all CTC-detected lesions ≥6 mm evaluated at OC was 91.6% (677 of 739) (Table 1). There were 62

Table 1 Positive predictive value for polyps detected at CTC screening Variablea

Positive predictive value (PPV)a

By-polyp assessment All polyps (≥6 mm) 91.6% According to lesion size Small (6–9 mm) 90.1% Large (≥10 mm) 92.7% According to lesion morphology Sessile 92.5% Pedunculated 96.5% Flat 77.7% Mass (≥3 cm) 97.6% According to diagnostic confidence Level-3 94.7% Level-2 83.5% Level-1 63.0% According to neoplastic histology Small (6–9 mm) 51.4% Large (≥10 mm) 67.0% All polyps (≥6 mm) 57.5% By-patient assessment 6-mm threshold 92.3% 8-mm threshold 93.0% 10-mm threshold 93.1% 30-mm threshold 97.4% a

(677/739) (410/451) (267/288) (441/477) (139/144) (73/94) (40/41) (554/585) (106/127) (17/27) (232/451) (193/288) (425/739) (442/479) (306/329) (228/245) (38/39)

See text for statistical significance testing

total CTC false-positive results in 54 patients, of which 17 patients (31.5%) had at least one true-positive matching lesion, yielding an overall per-patient PPV of 92.3% (442 of 479) at the 6-mm size threshold. The by-patient PPV at the 6-mm, 8-mm, and 10-mm polyp size thresholds were all within 1% of each other (Table 1). The by-patient PPV for lesions ≥3 cm was slightly higher at 97.4% (38 of 39). The corresponding FPR per polyp and per patient at the 6mm threshold was 8.4% and 7.7%, respectively. By-polyp PPV according to lesion size category was 90.1% (410 of 451) for small (6–9 mm) polyps and 92.7% (267 of 288) for large lesions. The difference in PPV according to small versus large lesion size was not statistically significant (p=0.4). The PPV for individual masses measuring ≥3 cm was 97.6% (40 of 41), including 100% (10 of 10) for carpet lesions. According to morphology, the PPV for sessile lesions was 92.5% (441 of 477), the PPV for pedunculated lesions was 96.5% (139 of 144), and the PPV for flat lesions was 77.7% (73 of 94). The difference in PPV between polypoid lesions (i.e., sessile or pedunculated) and flat lesions was highly significant (p<0.0001). PPV according to diagnostic confidence was 94.7% (554 of 585) for the highest confidence level (= 3), 83.5% (106 of 127) for intermediate confidence (= 2), and 63.0% (17 of 27) for the lowest confidence level (= 1). The differences in PPV between the three levels were all statistically significant, most notably for level-3 versus level-1 or level-2 (p< 0.0001), but also for level-2 versus level-1 (p=0.03). Level-3 or level-2 diagnostic confidence accounted for over 96% of all lesions called at CTC. A clustering effect was present between flat morphology and low diagnostic confidence. Although flat lesions represented only 12.7% (94 of 739) of all CTC-detected lesions that were prospectively called, they accounted for 40.7% of all confidence level-1 lesions (11 of 27), 35.5% (22 of 62) of all false positives, and 51.6% (16 of 31) of all false positives with a confidence level of 1 or 2. At histologic evaluation, neoplasms (n=425; tubular adenomas > tubulovillous/villous adenomas > adenocarcinomas > other neoplasms) accounted for the majority of matched lesions (62.8%; 425 of 677), followed by hyperplastic polyps (n=149). Less common non-neoplastic lesions included lymphoid polyps, juvenile polyps, inflammatory polyps, vascular blebs, and normal mucosa (in a polypoid configuration), in addition to a host of less common etiologies. PPV according to neoplastic histology was 57.5% (425 of 739) for all lesions ≥6 mm, 51.4% (232 of 451) for 6–9 mm lesions, and 67.0% (193 of 288) for large lesions ≥10 mm.

Discussion In the artificial setting of a clinical validation trial, the sensitivity and specificity of CTC relative to OC can be

1654

calculated, assuming that all patients undergo both procedures. However, in actual clinical practice, these performance measures cannot be obtained because only positive CTC cases are referred to OC for polypectomy, precluding assessment of the false-negative and truenegative rates. Performance measures that can be obtained in routine clinical CTC practice include the test positive rate, the false-positive rate (FPR), and the positive predictive value (PPV). Therefore, these values will necessarily become important parameters for program quality metrics and for comparison with other clinical programs. Our results demonstrate a very high PPV for our clinical screening program, indicating strong concordance between positive findings at CTC and subsequent evaluation at OC. PPV for all CTC-detected non-diminutive polyps was 92% for individual lesions, as well as at the patient level, corresponding to false-positive rates of less than 10%. To our knowledge, this is the largest study investigating PPV for CTC-detected polyps. Perhaps most noteworthy is the fact that these PPV values are substantially increased compared with the two large published multi-center CTC screening trials [1, 3]. In the Department of Defense (DoD) screening trial [3], the per-patient PPV was 41% at the 6-mm threshold, similar to the PPV of 40% in the subsequent ACRIN trial [1]. However, it must be noted that these values are lowered in part by the fact that only neoplastic lesions (adenomas and cancers) were considered as a true-positive result. When all matched lesions are included, regardless of histology, the 6-mm per-patient PPV in the DoD trial increased to 59% [13], still well below our current results. Similarly, the IMPACT trial, which evaluated a symptomatic cohort, reported a perpatient PPV of 61.9%, but this result was derived specifically for identifying patients with at least on advanced neoplasm—not all lesions ≥6 mm [14]. One could reasonably question whether we are now just calling the more obvious lesions at CTC, perhaps accounting for the high PPV. This concern for the possibility of false negatives in our CTC screening program was actually the impetus behind our previous study comparing the rates of advanced neoplasia detection at CTC versus OC screening [2]. In this study, we found that CTC screening actually led to the removal of more advanced neoplasms, the main target for colorectal screening and prevention, despite the fact that only 8% of adults screened by CTC were referred to OC. In addition, the test positive rates for CTC and OC were similar at both the 6-mm (13%) and 10-mm (4–5%) size thresholds. These data are very reassuring and suggest that CTC screening is achieving a very high sensitivity and specificity in clinical practice. It is important to note that although a small subset of patients may elect to undergo CTC surveillance of small polyps, this does not directly affect the PPV for 6–9 mm lesions since these cases are not considered. Rather, it merely changes the relative proportion of small and large lesions going to OC. Because PPV

does not appear to significantly vary between small and large polyps, the overall PPV is also not substantially affected. When patients with a positive CTC are sent to OC, there are sometimes additional lesions found at OC that were not reported at CTC. However, in our experience, over 70% of these additional lesions are diminutive in size (5 mm or smaller) [15], which are generally not reported at CTC [5, 16]. A number of these diminutive polyps can be identified at prospective CTC, but we do not individually characterize and catalogue these lesions. However, in the setting of larger co-existing lesions, we may incidentally mention their likely presence to the endoscopist. In terms of histology, over 70% of all additional polyps seen at OC that were not called at CTC are hyperplastic in our experience [15]. Furthermore, less than 3% of additional lesions have proven to be large adenomas, without any additional cancers uncovered to date. We evaluated several factors in order to determine their effect on PPV. Both polyp morphology and diagnostic confidence had a strong influence on PPV, whereas small versus large lesion size did not. The uniform PPV results for small and large lesions at both by-polyp and by-patient assessments are striking and reinforces how continued improvements in CTC technique and interpretation have improved performance down to the 6-mm size threshold. Perhaps not surprisingly, a polypoid morphology (i.e., sessile or pedunculated) was associated with a significantly higher PPV compared with flat lesions. A higher diagnostic confidence by the interpreting radiologist was also associated with a significantly higher PPV. Similar trends were noted in the DoD trial [12]. Importantly, although flat lesion morphology and low diagnostic confidence tended to cluster together, they both accounted for a small minority of each classification schema. In addition, although true flat lesions are less conspicuous at both CTC and OC, they actually tend to be much less histologically aggressive compared with polypoid lesions in our screening population [11]. Perhaps the common misperception that flat lesions are more ominous than sessile or pedunculated polyps has resulted in a tendency to overcall subtle potential flat lesions at CTC. Alternatively, it is also possible that some unmatched CTC-detected flat lesions are being missed at subsequent OC. In contrast, large flat masses (carpet lesions) had a high concordance at CTC–OC evaluation. It is interesting that small (6–9 mm) versus large (≥10 mm) lesion size had relatively little impact on PPV. In the DoD trial, the overall per-patient PPV was 67% at the 10-mm threshold, compared with 59% at the 6-mm threshold [13]. However, in the ACRIN trial, there was actually a decrease in per-patient PPV for larger adenoma sizes, dropping to 23% at the 10-mm threshold, compared with 40% at 6 mm [1]. This paradoxical result from the ACRIN trial is difficult to explain, but could potentially be related to overcalling of large lesions in order to maintain a

1655

reasonable detection sensitivity. Regardless, the high performance of CTC for small lesions in the current study is likely multi-factorial, related in part to incremental improvements in bowel preparation, colonic distention, multi-detector CT scanners, and the CTC software capabilities. Increased experience of the interpreting radiologists must also be considered as an important potential contributing factor. However, if quality initiatives in regards to training, technique, and performance results are uniformly applied, we believe that our results are generalizable to a wide variety of radiology practice settings. Demanding a PPVof 90% is probably not realistic, as this concordance level is dependent upon excellent performance at both CTC and OC, which may not always be attainable. Nonetheless, we believe a PPV on the order of 70–80% is achievable, which is considerably higher than the published validation trials. If the results for PPV in our clinical practice can be reproduced by others, this would have a dramatic impact upon the overall clinical efficacy and cost-effectiveness of this screening tool. Such information may help to convince late-adopting physicians to consider CTC as a viable screening option for their patients. Even though a number of prior cost-effectiveness analyses have concluded that CTC is cost-effective [17–19], the input assumptions for CTC performance were much more pessimistic compared with what we see in actual practice. Because some discrepant (unmatched) lesions will actually represent colonoscopy misses and not CTC overcalls, the term “CTC–OC concordance rate” may be more suitable than PPV. A CTC false-positive result was defined as any CTC-detected lesion without a matching OC correlate, which can result from either a CTC false positive or an OC false negative. Even though the endoscopist is unblinded to the CTC results in clinical practice, there will still be some real lesions that cannot be found at OC. In the DoD or ACRIN trials, such instances would have been unavoidably mislabeled as CTC false positives. The overall effect of these OC false-negative cases upon the CTC PPV

is probably relatively small. For all discordant cases, we review the original CTC findings to determine if the finding remains concerning for a possible OC false negative. In such cases, we generally arrange for a repeat CTC examination, with variable time interval that depends upon the relative importance of the finding. We did not directly address this issue in the current study but plan to systematically investigate this in a future project, after all further work-up has been completed. An additional potential issue is difficulty in lesion matching between CTC and OC [20], but we have found this to be less problematic in clinical practice compared with the artificial trial setting, perhaps due in part to the unblinded nature of endoscopy following positive CTC. We have found that the prospective diagnostic confidence level of the interpreting radiologist is a useful consideration in clinical practice. For example, communicating the possibility of a lower confidence level for a given CTC-detected lesion to the endoscopist may prevent an unnecessarily prolonged investigation if the lesion cannot be found. Alternatively, when we communicate a high diagnostic confidence level for a given CTC-detected lesion, the endoscopist will be more motivated to prolong the search. This is especially important for lesions located in relative blind spots for OC, most notably the back sides of folds within the right colon [21]. In summary, the overall per-polyp and per-patient PPV was 92% for non-diminutive CTC-detected lesions in our clinical screening program. An increased diagnostic confidence level and a polypoid (non-flat) morphology both correlated with a higher PPV, whereas small versus large polyp size had relatively little effect. If these results can be widely reproduced by other practices, CTC may become a more appealing primary screening option for patients, referring physicians, and third-party payers. Acknowledgements Dr. Pickhardt and Dr. Kim are consultants for Medicsight and Viatrox and are co-founders of VirtuoCTC.

References 1. Johnson CD, Chen MH, Toledano AY et al (2008) Accuracy of CT colonography for detection of large adenomas and cancers. N Engl J Med 359:1207–1217 2. Kim DH, Pickhardt PJ, Taylor AJ et al (2007) CT colonography versus colonoscopy for the detection of advanced neoplasia. N Engl J Med 357:1403–1412

3. Pickhardt PJ, Choi JR, Hwang I et al (2003) Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults. N Engl J Med 349:2191–2200 4. Taylor SA, Laghi A, Lefere P, Halligan S, Stoker J (2007) European society of gastrointestinal and abdominal radiology (ESGAR): consensus statement on CT colonography. Eur Radiol 17:575– 579 5. Zalis ME, Barish MA, Choi JR et al (2005) CT colonography reporting and data system: a consensus proposal. Radiology 236:3–9

6. Mavranezouli I, East JE, Taylor SA (2008) CT colonography and costeffectiveness. Eur Radiol. doi:10.1007/ s00330-008-1058-y 7. Laghi A (2005) Virtual colonoscopy: clinical application. Eur Radiol 15 Suppl 4:D138–141 8. Pickhardt PJ (2005) Virtual colonoscopy: issues related to primary screening. Eur Radiol 15: D133–D137

1656

9. Pickhardt PJ, Lee AD, McFarland EG, Taylor AJ (2005) Linear polyp measurement at CT colonography: in vitro and in vivo comparison of two-dimensional and three-dimensional displays. Radiology 236:872–878 10. Dachman AH, Zalis ME (2004) Quality and consistency in CT colonography and research reporting. Radiology 230:319–323 11. Pickhardt PJ (2009) CT colonography: time for clinical implementation. Nat Rev Clin Oncol 6:187–188 12. Pickhardt PJ, Choi JR, Nugent PA, Schindler WR (2004) The effect of diagnostic confidence on the probability of optical colonoscopic confirmation of potential polyps detected on CT colonography: prospective assessment in 1,339 asymptomatic adults. AJR Am J Roentgenol 183:1661–1665

13. Pickhardt PJ (2005) Limitations of virtual colonoscopy - in response. Ann Intern Med 142:155–155 14. Regge D, Laudi C, Galatola G et al (2009) Diagnostic accuracy of computed tomographic colonography for the detection of advanced neoplasia in individuals at increased risk of colorectal cancer. JAMA 301:2453–2461 15. Cornett D, Barancin C, Roeder B et al (2008) Findings on optical colonoscopy after positive CT colonography exam. Am J Gastroenterol 103:2068–2074 16. Pickhardt PJ (2007) Screening CT colonography: how I do it. AJR Am J Roentgenol 189:290–298 17. Hassan C, Pickhardt P, Laghi A et al (2008) Computed tomographic colonography to screen for colorectal cancer, extracolonic cancer, and aortic aneurysm. Arch Intern Med 168:696– 705 18. Pickhardt PJ, Hassan C, Laghi A, Zullo A, Kim DH, Morini S (2007) Costeffectiveness of colorectal cancer screening with computed tomography colonography: the impact of not reporting diminutive lesions. Cancer 109:2213–2221

19. Vijan S, Hwang I, Inadomi J et al (2007) The cost-effectiveness of CT colonography in screening for colorectal neoplasia. Am J Gastroenterol 102:380–390 20. Liedenbaum MH, de Vries AH, Halligan S et al (2009) CT colonography polyp matching: differences between experienced readers. Eur Radiol 19:1723–1730 21. Pickhardt PJ, Nugent PA, Mysliwiec PA, Choi JR, Schindler WR (2004) Location of adenomas missed by optical colonoscopy. Ann Intern Med 141:352–359