Valid and reliable self-report measures of cancer screening behaviors are important for evaluating efforts to improve adherence to guidelines. We evaluated test-retest reliability and validity of self-report of the fecal occult blood test (FOBT), sigmoidoscopy (SIG), colonoscopy (COL), and barium enema (BE) using the National Cancer Institute colorectal cancer screening (CRCS) questionnaire. A secondary objective was to evaluate reliability and validity by mail, telephone, and face-to-face survey administration modes. Consenting men and women, 51 to 74 years old, receiving care at a multispecialty clinic for at least 5 years who had not been diagnosed with colorectal cancer were stratified by prior CRCS status and randomized to survey mode (n = 857). Within survey mode, respondents were randomized to complete a second survey at 2 weeks, 3 months, or 6 months. Comparing self-report with administrative and medical records, concordance estimates were 0.91 for COL, 0.85 for FOBT, 0.85 for SIG, and 0.92 for BE. Overall sensitivity estimates were 0.91 for COL, 0.82 for FOBT, 0.76 for SIG, and 0.56 for BE. Specificity estimates were 0.91 for COL, 0.86 for FOBT, 0.89 for SIG, and 0.97 for BE. Sensitivity and specificity varied little by survey mode for any test. Report-to-records ratio showed overreporting for SIG (1.1), COL (1.15), and FOBT (1.57), and underreporting for BE (0.82). Reliability at all time intervals was highest for COL; there was no consistent pattern according to survey mode. This study provides evidence to support the use of the National Cancer Institute CRCS questionnaire to assess self-report with any of the three survey modes.