Tyrosine kinase inhibitors (TKIs) are very effective against non-small cell lung cancer (NSCLC) caused by epidermal growth factor receptor (EGFR) mutation. Before the approval of osimertinib in March 2016, there were only three available EGFR TKIs (gefitinib, erlotinib, and afatinib) for the therapy of NSCLC in Japan. Osimertinib can be indicated only against T790M+ lung cancer as a second-line therapy. However, whether gefitinib, erlotinib, or afatinib is most appropriate as a first-line therapy is still a controversial issue. The aim of this study was to compare the effectiveness of gefitinib, erlotinib, and afatinib. We retrospectively reviewed the records of 310 patients with the diagnosis of EGFR mutation-associated NSCLC including 147 patients treated with EGFR TKIs. Time to treatment failure and overall survival were evaluated. There were no significant differences in time to treatment failure (gefitinib: 9.2 months; erlotinib: 9.8 months; afatinib: 13.1 months) and overall survival (gefitinib: 27.3 months; erlotinib: 29.3 months; afatinib data not available) among NSCLC patients treated with the three different EGFR TKIs. Subgroup analysis showed that smoking status has a significant influence on both time to treatment failure and overall survival. In conclusion, this study showed comparable clinical efficacy of gefitinib, erlotinib, and afatinib in Japanese patients with NSCLC.