Eur Radiol DOI 10.1007/s00330-016-4705-8

CHEST

Interval lung cancer after a negative CT screening examination: CT findings and outcomes in National Lung Screening Trial participants David S. Gierada 1 & Paul F. Pinsky 2 & Fenghai Duan 3 & Kavita Garg 4 & Eric M. Hart 5 & Ella A. Kazerooni 6 & Hrudaya Nath 7 & Jubal R. Watts Jr 7 & Denise R. Aberle 8

Received: 18 August 2016 / Revised: 3 November 2016 / Accepted: 15 December 2016 # European Society of Radiology 2017

Abstract Objectives This study retrospectively analyses the screening CT examinations and outcomes of the National Lung Screening Trial (NLST) participants who had interval lung cancer diagnosed within 1 year after a negative CT screen and before the next annual screen. Methods The screening CTs of all 44 participants diagnosed with interval lung cancer (cases) were matched with negative CT screens of participants who did not develop lung cancer (controls). A majority consensus process was used to classify each CT screen as positive or negative according to the NLST criteria and to estimate the likelihood that any abnormalities detected retrospectively were due to lung cancer. Results By retrospective review, 40/44 cases (91%) and 17/44 controls (39%) met the NLST criteria for a positive screen (P < 0.001). Cases had higher estimated likelihood of lung cancer (P < 0.001). Abnormalities included pulmonary nodules ≥4 mm (n = 16), mediastinal

(n = 8) and hilar (n = 6) masses, and bronchial lesions (n = 6). Cancers were stage III or IV at diagnosis in 32/44 cases (73%); 37/44 patients (84%) died of lung cancer, compared to 225/649 (35%) for all screendetected cancers (P < 0.0001). Conclusion Most cases met the NLST criteria for a positive screen. Awareness of missed abnormalities and interpretation errors may aid lung cancer identification in CT screening. Key points • Lung cancer within a year of a negative CT screen was rare. • Abnormalities likely due to lung cancer were identified retrospectively in most patients. • Awareness of error types may help identify lung cancer sooner.

Keywords Lung cancer . Mass screening . Computed tomography . Diagnostic errors . False-negative reactions

Electronic supplementary material The online version of this article (doi:10.1007/s00330-016-4705-8) contains supplementary material, which is available to authorized users. * David S. Gierada [email protected]

1

Mallinckrodt Institute of Radiology, Washington University School of Medicine, Box 8131, 510 S. Kingshighway Blvd., St. Louis, MO 63110, USA

2

National Cancer Institute, 9609 Medical Center Drive, Room 5E108, Bethesda, MD 20892, USA

3

Department of Biostatistics and Center for Statistical Sciences, Brown University School of Public Health, Providence, RI 02912, USA

4

University of Colorado School of Medicine, Mail Stop F726, Box 6510, Aurora, CO 80045, USA

5

Department of Radiology, Northwestern University Feinberg School of Medicine, 676 N. Saint Clair, Suite 800, Chicago, IL 60611, USA

6

Department of Radiology, University of Michigan Health System, Cardiovascular Center, Room #5482, 1500 East Medical Center Drive, Ann Arbor, MI 48109, USA

7

Department of Radiology-JTN370, University of Alabama at Birmingham School of Medicine, 619 19th Street South, Birmingham, AL 35233, USA

8

Department of Radiological Sciences, David Geffen School of Medicine at UCLA, 924 Westwood Boulevard, Suite 420, Los Angeles, CA 90024, USA

Eur Radiol

Introduction In low-dose CT screening for lung cancer, much attention has been directed at the relatively high frequency of small pulmonary nodule detection and the feasibility of reducing the number of unnecessary workups for those that have a very low rate of malignancy [1–6]. Less commonly, CT screens may be interpreted as not suspicious for lung cancer when lung cancer is present. In the National Lung Screening Trial (NLST), 44 of the 1060 (4.2%) CT screening arm participants who developed lung cancer had an interval cancer, in which the diagnosis was made within 1 year of a CT screen that had been classified as not suspicious for lung cancer, and before the next annual screen [1]. These interval cancers may have been associated with CT interpretation errors or, alternatively, they may have been undetectable by CT or may have not developed until after the screening CT. We analysed the CT screens associated with the interval cancers occurring in the NLST to determine whether abnormalities potentially related to lung cancer were present in retrospect, to evaluate the nature of abnormalities missed or misinterpreted, and to assess clinical outcomes in these cases relative to the NLST overall.

locations were recorded in the NLST database by anatomic site: lobe of lung, right or left hilum, or mediastinum. Postscreening imaging and other diagnostic tests were performed outside of the NLST protocol, so it was not possible to directly map specific screening CT abnormalities to the diagnosed malignancy; as such, we also assessed the likelihood that any positive screening abnormalities were due to lung cancer. The screening CT scans of the 44 cases were combined with 44 randomly selected NLST CT scans reported as negative by the original NLST radiologist and not associated with a diagnosis of lung cancer during the trial (controls). Controls were matched with cases by age, gender, current smoking status, scanner model, and study year (18 baseline screens, 10 first annual screens, 16 second annual screens). Although this design did not replicate the proportion of screens associated with lung cancer in the NLST, control scans were included to reduce bias toward overcalling abnormalities in cases and for additional reference in comparing interpretations of study readers and original NLST radiologists in this setting designed to enhance detection sensitivity. The image file folders of the 44 cases were named with the anatomic locations of the lung cancers; the image file folders of the corresponding controls were named with the same anatomic locations (Table 1), and the presentation order of the 88 CT screens was randomised. Study readers reviewed CT screens knowing the nature of the study, but without knowing which scans were of cases and which of controls.

Methods Subjects The NLST enrolled 53,454 participants aged 55-74 years with a smoking history of at least 30 pack-years, including both current smokers and those who had quit within the past 15 years [7]. Participants were randomised to screening with either low-dose CT scans or posteroanterior chest radiographs three times at annual intervals at 33 screening centres across the USA. NLST radiologists used standardised reporting forms and coded lists to record abnormalities, classify screening examinations as positive or negative, and make follow-up recommendations, and they also were able to enter comments as written text (see Electronic Supplementary Material for details). After a median participant follow-up time of 6.5 years, lung cancer mortality was 20% lower in the CT arm [1]. All participants were enrolled and consented according to protocols approved by the institutional review boards of the enrolling centres. We reviewed de-identified screening CT scans of the 44 NLST participants with interval lung cancers, defined as cancers diagnosed within 1 year of a negative CT screen and before their next annual screen (cases) to determine whether abnormalities were present in retrospect that met the NLST criteria for a positive screening examination. Lung cancer

Table 1 Locations of lung cancer histopathologic diagnoses recorded in NLST database Cancer location

Number of cases

Right upper lobe Right middle lobe Right lower lobe Left upper lobe

13 4 5 3

Left lower lobe Right hilum

6 4

Left hilum

3

Carina Mediastinum Other Unknown All

Additional locations

Mediastinum (1 case) Left lower lobe (1 case) Left hilum and carina (1 case) Mediastinum (2 cases) Right main bronchus (1 case) Left main bronchus (2 cases)

1 1 1 3 44

Case and corresponding control image file folders were named with all recorded cancer locations

Eur Radiol

Readings Overview We reviewed the CT scans in two phases. The goal of the first review (abnormality detection round) was to identify all abnormalities potentially due to lung cancer. The abnormalities documented in the detection round and their anatomic and CT slice locations were provided for the second review (interpretation round), which was done to (1) classify each scan as positive or negative according to the NLST criteria, (2) identify the most suspicious abnormality, and (3) estimate the likelihood that cancer was present for each case.

criteria. For each screen, the readers indicated which of the listed abnormalities was most important in making their classification. Readers also noted whether the positive abnormality was in the recorded cancer location, whether malignancy other than lung cancer (e.g., primary mediastinal neoplasm or metastatic disease from an extrathoracic primary) was a consideration in addition to lung cancer; and whether any nodules <4 mm were present (Electronic Supplementary Material Table 1). The level of suspicion that lung cancer was present based on the aforementioned 5-point scale, and a single estimated per cent probability of lung cancer were again estimated by each reader for each screen.

Abnormality detection round

Classification of screens

The 88 CT screens were reviewed independently by 5 thoracic radiologists (DSG, KG, EMH, HN, JRW), 4 of whom had participated in the NLST. The readers were instructed to record any findings meeting the NLST criteria for a positive screen (i.e., noncalcified nodule ≥4 mm in greatest transverse dimension or other abnormalities potentially due to lung cancer, including hilar or mediastinal lymph node enlargement and pleural effusion) and to record noncalcified nodules <4 mm in the lobe of cancer diagnosis (Electronic Supplementary Material Figure 1). Readers rated their level of suspicion that the observed abnormalities were due to lung cancer using a 5-point scale that combined their estimated probability of lung cancer with the type of follow-up they would recommend based on their own knowledge and experience: 1 = none (0% probability; no follow-up needed); 2 = very low (1-5% probability; 12-month CT); 3 = moderate (6-10% probability; 6-month CT); 4 = high (1130% probability; 3-month CT); 5 = very high (>30% probability; <3 month CT , PET, biopsy, or contrast CT). In order to optimally capture independent levels of suspicion, readers also separately recorded their estimated probability that lung cancer was present on a continuous scale from 0 to 100%.

Cases and controls were given a final classification as consensus-positive or consensus-negative screens if there was agreement by four or more readers in the interpretation round. Cases and controls designated as positive by three readers and negative by three readers were considered equivocal. Consensus-positive cases were considered positive in the known cancer location if at least four readers considered the case positive in the known cancer location. The mean and median suspicion levels on the 5-point scale and per cent probabilities on the continuous scale were calculated across all readers for each case and control.

Interpretation round

Statistical analysis

The abnormalities (including anatomic and CT slice locations, suspicion levels, and per cent probability of lung cancer) recorded by all readers for each screen in the abnormality detection round were entered into electronic spread sheets, which were distributed to all first round readers and to an additional thoracic radiologist who had also been an NLST reader (EAK). The six readers referred to these lists of abnormalities while independently re-reviewing the 88 screens and classified each screen as positive or negative according to the NLST

The chi-squared test was used to compare the proportion of consensus-positive subjects in the case and control groups and to compare the proportion of stage III/ IV subjects and lung cancer mortality rates of the cases with those of all NLST subjects who had CT screendetected lung cancer. The Wilcoxon rank sum test was used to compare the suspicion level and lung cancer probability estimates of the case and control groups. Statistical tests were performed with SAS version 9.3.

Most important abnormality An abnormality cited as the most suspicious by at least three readers was considered the most important abnormality in each case.

Clinical outcomes The cancer stage at diagnosis, vital status, and cause of death were reviewed for all cases.

Eur Radiol Table 3

Results

Most important abnormalities in lung cancer cases

Abnormality by study readers

Classification of screens On final retrospective classification (Table 2), 40 cases were consensus-positive screens and 4 cases were consensus negative; in contrast, 17 controls were consensus positive, 23 consensus negative, and 4 equivocal (P < 0.0001). The quantitative suspicion levels using both the 5-point scale and the continuous probability scale were much higher in cases than controls (P < 0.0001) (Table 2). Cases were positive in the known cancer location in 31 of the 36 (86%) consensus-positive cases in which the cancer location was known; the corresponding mean suspicion level was 4.1 ± 1.1 and mean estimated probability of lung cancer was 56% ± 36 (2-16% in 10 cases, 44-68% in 5 cases, and 70-98% in 16 cases). In consensus-positive cases with suspicious findings outside the known cancer location (5 cases) or in which the cancer location was unknown (4 cases), the mean suspicion level was 3.3 ± 1.4 and the mean estimated probability of lung cancer was 35% ± 35 (2-28% in 6 and 65-92% in 3).

Total number of cases (N = 44)

Abnormality recorded by original readers (N = 18)

Lung nodule ≥ 10 mm

8

3

4-9 mm

8

2

1 8

3* 5

6 6

2

< 4 mm Mediastinal mass or adenopathy Hilar mass or adenopathy Bronchial lesion None Nodular thickening along wall of bulla Localised area of ground glass, interstitial thickening, and cystic lucency Left upper lobe collapse with cut-off bronchus Focal consolidation

3 1 1

1

1

1

1

1

*In these cases, the study readers recorded a 4- or 5-mm nodule as the most important abnormality

Most important abnormalities (Table 3) The same abnormality was cited as most important by all 6 readers in 26 cases, by 5 readers in 8 cases, by 4 readers in 8 cases, and by 3 readers in 2 cases. A pulmonary nodule ≥4 mm was the most important abnormality in 16 of 44 cases (Fig. 1). Eleven of these nodules were in the peripheral region of the lung, two were near the

Table 2

Summary of readings

Study read Consensus positive Consensus negative Equivocal Total Suspicion of lung cancer* Mean ± SDa Median Probability of lung cancer Mean ± SDa Median Range

Cases

Controls

40 4 0 44

17 23 4 44

3.7 ± 1.4 4.3 47% ± 38 49% 0.3-98%

1.8 ± 0.6 1.8 3.0% ± 4.1 1.5% 0.0-16%

*Based on a 5-point scale (1 = lowest to 5 = highest; see Methods for full explanation) a

P < 0.0001 for difference between cases and controls, Wilcoxon rank sum test

hilum (Fig. 1c and d), four were juxtavascular, and three were associated with a cystic airspace. A mediastinal mass or lymph node enlargement (Fig. 2) was reported in eight cases, a hilar mass or lymph node enlargement (Fig. 3) and a bronchial lesion (Fig. 4) in six cases each, and unique abnormalities (Fig. 5) in four cases. In 22 of the 40 consensus-positive cases, the original NLST radiologist did not record the most important abnormality reported by the study readers. In the remaining 18 consensuspositive cases, the original radiologist recorded the most important abnormality reported by the study readers, including 8 with nodules and 7 with a mediastinal or hilar mass or adenopathy (Table 3). Fourteen of these 18 cases were classified by the original radiologist as a negative screen with clinically significant abnormalities not suspicious for lung cancer, and a recommendation for clinical follow-up was given. At least one study reader also considered a malignancy other than lung cancer to be a possibility in 6 of these 14 cases. In 4 of these 14 cases (4 of the 8 with nodules), the original reader recorded the same nodule that was considered the most important abnormality by the study readers, but attributed the cause to a non-neoplastic aetiology. The other 4 of these 18 consensus-positive cases had small nodules reported on both original and study reads. In three of these cases a nodule was measured as less than the 4-mm threshold for a positive screen by the original radiologist and ≥4 mm by the study readers. The fourth case involved a 5-6mm nodule that had been classified by the original radiologist

Eur Radiol

Discussion In the NLST, interval lung cancer diagnosed within the year following a negative CT screen and before the next annual screen was rare; the 44 cases represented 0.08% of the 56,980 negative CT screens over the 3 screening rounds, or 7.7 per 10,000 negative screens. Our retrospective analysis found a consensus-negative interpretation for only four of these interval cancer cases, for a rate of interval cancer with no retrospectively positive findings by the NLST criteria of 0.007%, or 0.7 per 10,000 negative screens. There was only one consensus-negative case in which no abnormality was noted by any reader in the interpretation round, a rate of 0.0018%, or 0.18 per 10,000 negative screens. Under the

Fig. 1 Pulmonary nodules (arrows) identified as most important abnormality by consensus on retrospective review. a Original screening result was ‘negative-minor abnormality’ with reader comment Bnew right upper lobe abnormality, probably inflammatory or trivial atelectasis^; study readers’ mean suspicion = 4.7, mean probability of lung cancer = 47%. Diagnosis was carcinoma, not otherwise specified. b Original screening result was ‘negative-no significant abnormality’; study readers’ mean suspicion = 2.5, mean probability of lung cancer = 8%. Diagnosis was adenocarcinoma. c Original screening result was ‘negative-no significant abnormality’; study readers’ mean suspicion = 5.0, mean probability of lung cancer = 98%. Diagnosis was adenocarcinoma. d Original screening result was ‘negative-minor abnormality’ (coronary artery calcification); study readers’ mean suspicion = 4.7, mean probability of lung cancer = 75%. Diagnosis was squamous cell carcinoma

as negative because it had been stable for at least 2 years, as allowed by the NLST protocol; historical images were not available to the study readers. Of the four consensus-negative cases, there was only one in which no abnormalities were found by any of the six readers. In two of the consensus-negative cases, four of the readers cited no abnormalities and two cited low suspicion abnormalities in the cancer lobe with probabilities ranging 1-5%. In the fourth case, a nodule <4 mm in the cancer lobe was the most important abnormality. Clinical outcomes Of the 44 cases, 32 (73%) were stage III or IV at diagnosis, compared to 30% (189/635, not including 14 of unknown stage) of all screen-detected lung cancers in the NLST (P < 0.0001). The lung cancer mortality rate of the 44 cases was 80% (35/44), compared to 35% (225/649) for all screen-detected cancers within the NLST follow-up period (P < 0.0001).

Fig. 2 Mediastinal masses and lymphadenopathy (arrows) identified as most important abnormality by consensus on retrospective review. a Original screening result was ‘negative-minor abnormality’ (nodule <4 mm) with reader comment Bscar^; study readers’ mean suspicion = 3.2, mean probability of lung cancer = 28%. Diagnosis was squamous cell carcinoma. b Original screening result was ‘negative-significant abnormality not suspicious for lung cancer’, with reader comment on interval increase in size of mediastinal lymph nodes and recommendation for 3-6-month follow-up CT; study readers’ mean suspicion = 3.2, mean probability of lung cancer = 14%; two study readers thought non-lung cancer should be considered. Diagnosis was adenocarcinoma. c Original screening result was ‘negative-significant abnormality not suspicious for lung cancer’ (hilar/mediastinal adenopathy or mass ≥10 mm); study readers’ mean suspicion = 5.0, mean probability of lung cancer 65%; 3 study readers thought non-lung cancer should be considered. Diagnosis was non-small cell carcinoma. d Original screening result was ‘negative-significant abnormality not suspicious for lung cancer’ (hilar/mediastinal adenopathy or mass ≥10 mm), with reader comment on aggressive appearance extending into right upper lobe; study readers’ mean suspicion = 5.0, mean probability of lung cancer 92%; two study readers thought non-lung cancer should be considered. Diagnosis was non-small cell carcinoma

Eur Radiol

Fig. 3 Hilar masses (arrows) identified as most important abnormality by consensus on retrospective review. a Original screening result was ‘negative-minor abnormality’ (coronary artery calcification); study readers’ mean suspicion = 5.0, mean probability of lung cancer = 86%. Diagnosis was adenocarcinoma. b Original screening result was ‘negative-minor abnormality’ (hilar/mediastinal adenopathy or mass ≥ 10 mm); study readers’ mean suspicion = 4.2, mean probability of lung cancer = 68%. Diagnosis was large cell neuroendocrine carcinoma

screen classification guidelines of the American College of Radiology developed after the NLST (Lung-RADS), which increase the nodule size threshold for interval diagnostic evaluation to 6-mm average diameter, the interval cancer rate in NLST would have been higher as a small number of cancers in NLST were found after screens that were positive because of nodules of 4 or 5 mm diameter [5, 6]. In 22 of the 40 consensus-positive cases, the original NLST radiologists did not record the most important abnormality reported by study readers, suggesting that the abnormality was most likely overlooked; less likely, the abnormality may have been observed but fully discounted. In another 14 consensus-positive cases, the original NLST radiologists classified the screen as negative for lung cancer, but identified abnormalities for further evaluation; in 6 of these, some

Fig. 4 Bronchial lesions (arrows) identified as most important abnormality by consensus on retrospective review. a Original screening result was ‘negative-minor abnormality’ (3 right middle lobe nodules, largest 5 mm, examination called negative after comparison to prior CT); study readers’ mean suspicion = 3.8, mean probability of lung cancer = 53%. Diagnosis was adenocarcinoma. b Original screening result was ‘negative-minor abnormality’ (noncalcified nodules of 4 mm in right upper lobe and 7 mm in left lower lobe, examination called negative after comparison to prior CT); study readers’ mean suspicion = 5.0, mean probability of lung cancer = 94%. Diagnosis was squamous cell carcinoma

readers thought non-lung malignancy also was possible. Given the types of abnormalities in these cases (Table 3), lung cancer should be considered when hilar or mediastinal or juxtamediastinal masses are present without lung nodules, when cystic airspaces are associated with nodules or wall thickening, and with focal consolidation. The most frequent of the retrospectively identified abnormalities shown in Table 3 was a noncalcified lung nodule, which is the primary detection task of lung cancer screening. In 9 of these 17 cases (53%) the nodule was less than 10 mm in diameter, and most were peripheral. It is well known that pulmonary nodules, including those larger than 1 cm representing lung cancer, can be missed on CT scans [8, 9]. Many of the abnormalities identified here retrospectively were substantial, while others were relatively subtle endobronchial lesions, mediastinal lymph node enlargement, and hilar or juxtamediastinal lesions that may be challenging to detect on non-contrast CT. Although the CT screens were acquired at nominal 2.5-mm or smaller slice thickness, no technologies known to improve nodule detection were used in the NLST, such as sliding-slab maximum intensity projections [10, 11], volume-rendering [12] or computer-aided detection software [13, 14]. In a smaller screening trial, missed cancers were found to be predominantly subtle intrapulmonary nodules, sometimes obscured by overlapping vessels or resembling other disease [15]. In the NELSON trial, which allowed use of nodule detection software, a suspicious abnormality was retrospectively identified in 22 of 34 (65%) interval cancers [16], compared to 40 of 44 interval cancers (91%) with positive findings by the NLST criteria in our study; missed findings in NELSON were primarily nodules, bronchial lesions, thickened walls of bullae, and lymph node enlargement [17].

Fig. 5 Other lesions (arrows) identified as most important abnormality by consensus on retrospective review. a Soft tissue thickening at edge of bulla. Original screening result was ‘negative-significant abnormality not suspicious for lung cancer’ (9 mm left lower lobe nodule, emphysema, other not specified; examination called negative after comparison to prior CT); study readers’ mean suspicion = 5.0, mean probability of lung cancer = 95%. Diagnosis was pseudoglandular squamous cell carcinoma. b Focal consolidation. Original screening result was ‘negative-significant abnormality not suspicious for lung cancer’ (consolidation); study readers’ mean suspicion = 4.8, mean probability of lung cancer = 71%; 1 study reader thought non-lung cancer should be considered. Diagnosis was mucinous adenocarcinoma

Eur Radiol

The 80% lung cancer mortality rate of the 44 interval cancers was higher than the 35% rate for all screen-detected cancers in the NLST, suggesting that the interval cancers may have been more aggressive or of higher pathological grade. The clinical events that led to the diagnosis of lung cancer in the NLST were not consistently documented, but in those cases interpreted as negative with clinically significant abnormalities not suspicious for lung cancer, the recommendations for further diagnostic evaluation may have led to the diagnosis. Interval cancers in the NELSON trial also were more advanced, with 29 of 35 (83%) stage III or IV at diagnosis compared to 44 of 196 (22%) screen-detected cancers (P < 0.0001) [16]; mortality was not reported. This multi-reader, multi-phased retrospective review was purposely designed to be more sensitive to the detection of abnormalities than original NLST reads, as seen by a 39% (17/44) positivity rate for the control scans. This is multifactorial and reflects the artificial setting of retrospective review, the high proportion of cases relative to the normal screening setting, and the heightened sensitivity of readers to finding abnormalities in the cancer locations labelled in both cases and controls. However, both suspicion levels and probabilities of lung cancer in controls were far lower than in cases, with no control subjects given a probability greater than 16%. A limitation of this study is the lack of access to the postscreening imaging and other tests performed outside of the NLST and the consequent uncertainty as to whether all retrospectively identified abnormalities represent the diagnosed lung cancers; because of this, the number of interval cancers that had no detectable abnormalities at CT screening could be greater than the one case identified here. We also note that with the variability in performance documented among NLST readers [18–20], there were undoubtedly unrecorded positive screening abnormalities in other NLST participants who did not develop an interval cancer, which were then reported on a subsequent screen that led to the diagnosis of lung cancer; thus, we cannot estimate the missed cancer rate for the entire NLST population. In summary, interval lung cancer in the NLST was rare, but most cases had abnormalities identifiable retrospectively that met the NLST criteria for a positive screen and high probability of representing the subsequently diagnosed cancers. The presentation relatively soon after the negative screen, advanced stage, and high mortality of these cancers suggest they were more aggressive or higher grade tumours than the screen-detected cancers and may not have been impacted by earlier diagnosis. Nevertheless, awareness of the types of missed or misinterpreted abnormalities in this cohort may help to avoid delays in diagnosis and treatment of lung cancer. In particular, CT screens lacking a lung nodule should be reviewed for mediastinal, juxtamediastinal, hilar, and endobronchial abnormalities that may represent cancer, and

cystic lesions should be closely evaluated for associated nodules or wall thickening. Acknowledgments The National Lung Screening Trial was supported by the following NIH grants and contracts: U01-CA-80098, U01-CA79778, N01-CN-25522, N01-CN-25511, N01-CN-25512, N01-CN25513, N01-CN-25514, N01-CN-25515, N01-CN-25516, N01-CN25518, N01-CN-25524, N01-CN-75022, N01-CN-25476, and N02CN-63300. Data for this study were obtained through the Cancer Data Access System of the National Cancer Institute (https://biometry.nci.nih. gov/cdas/). We thank Joshua Rathmell and Brett Thomas of Information Management Services for assistance with data retrieval. The scientific guarantor of this publication is DSG. The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article. Two of the authors have significant statistical expertise. Institutional Review Board approval was not required because only previously acquired, de-identified, publicly available data were used. Written informed consent was obtained from all subjects (patients) in this study. Methodology: • retrospective • case-control study/observational • multicentre study

References 1.

2.

3.

4.

5.

6.

7.

8. 9.

10.

11.

Aberle DR, Adams AM, Berg CD et al (2011) Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 365:395–409 Humphrey LL, Deffebach M, Pappas M et al (2013) Screening for lung cancer with low-dose computed tomography: a systematic review to update the US Preventive services task force recommendation. Ann Intern Med 159:411–420 Harris RP, Sheridan SL, Lewis CL et al (2014) The harms of screening: a proposed taxonomy and application to lung cancer screening. JAMA Intern Med 174:281–285 Yip R, Henschke CI, Yankelevitz DF, Smith JP (2014) CT screening for lung cancer: alternative definitions of positive test result based on the national lung screening trial and international early lung cancer action program databases. Radiology 273:591–596 Gierada DS, Pinsky P, Nath H, Chiles C, Duan F, Aberle DR (2014) Projected outcomes using different nodule sizes to define a positive CT lung cancer screening examination. J Natl Cancer Inst 106 Pinsky PF, Gierada DS, Black W et al (2015) Performance of LungRADS in the National Lung Screening Trial: a retrospective assessment. Ann Intern Med 162:485–491 Aberle DR, Berg CD, Black WC et al (2011) The National Lung Screening Trial: overview and study design. Radiology 258:243– 253 Gurney JW (1996) Missed lung cancer at CT: imaging findings in nine patients. Radiology 199:117–122 White CS, Romney BM, Mason AC, Austin JH, Miller BH, Protopapas Z (1996) Primary carcinoma of the lung overlooked at CT: analysis of findings in 14 patients. Radiology 199:109–115 Diederich S, Lentschig MG, Overbeck TR, Wormanns D, Heindel W (2001) Detection of pulmonary nodules at spiral CT: comparison of maximum intensity projection sliding slabs and single-image reporting. Eur Radiol 11:1345–1350 Gruden JF, Ouanounou S, Tigges S, Norris SD, Klausner TS (2002) Incremental benefit of maximum-intensity-projection images on

Eur Radiol

12.

13.

14.

15.

observer detection of small pulmonary nodules revealed by multidetector CT. AJR Am J Roentgenol 179:149–157 Peloschek P, Sailer J, Weber M, Herold CJ, Prokop M, SchaeferProkop C (2007) Pulmonary nodules: sensitivity of maximum intensity projection versus that of volume rendering of 3D multidetector CT data. Radiology 243:561–569 Armato SG 3rd, Li F, Giger ML, MacMahon H, Sone S, Doi K (2002) Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology 225:685–692 Rubin GD, Lyo JK, Paik DS et al (2005) Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology 234:274–283 Li F, Sone S, Abe H, MacMahon H, Armato SG 3rd, Doi K (2002) Lung cancers missed at low-dose helical CT screening in a general population: comparison of clinical, histopathologic, and imaging findings. Radiology 225:673–683

16.

17.

18.

19.

20.

Horeweg N, Scholten ET, de Jong PA et al (2014) Detection of lung cancer through low-dose CT screening (NELSON): a prespecified analysis of screening test performance and interval cancers. Lancet Oncol 15:1342–1350 Scholten ET, Horeweg N, de Koning HJ et al (2015) Computed tomographic characteristics of interval and post screen carcinomas in lung cancer screening. Eur Radiol 25:81–88 Gierada DS, Pilgram TK, Ford M et al (2008) Lung cancer: interobserver agreement on interpretation of pulmonary findings at lowdose CT screening. Radiology 246:265–272 Singh S, Pinsky P, Fineberg NS et al (2011) Evaluation of reader variability in the interpretation of follow-up CT scans at lung cancer screening. Radiology 259:263–270 Pinsky PF, Gierada DS, Nath PH, Kazerooni E, Amorosa J (2013) National lung screening trial: variability in nodule detection rates in chest CT studies. Radiology 268:865–873