Cancer immunotherapy including immune checkpoint inhibitors (ICI) has revolutionized non-small cell lung cancer (NSCLC) therapy. Recently, the microbiota status "before" initiation of ICI therapy has been emphasized as a predictive biomarker in patients undergoing ICI therapy. However, the microbiota diversity and composition "during" ICI therapy is unknown. This multicenter, prospective observational study analyzed both saliva and feces from 28 patients with NSCLC. We performed 16S ribosomal RNA gene sequencing, then analyzed associations of oral and gut microbiota diversity or composition with ICI response. At the genus level, the alpha diversity of the gut microbiota was significantly greater in responders ( = 17) than in non-responders ( = 11) (Chao 1, = 0.0174; PD whole tree, = 0.0219; observed species, = 0.0238; Shannon, = 0.0362), while the beta diversity of the gut microbiota was significantly different (principal coordinates analysis, = 0.035). Compositional differences in the gut microbiota were observed between the two groups; in particular, g_ was enriched in responders, whereas o_ was enriched in non-responders. The progression-free survival (PFS) of patients enriched gut microbiota of g_ was significantly longer [median survival time (MST): not reached vs. 549 days, = 0.0480] and the PFS of patients with gut microbiota of o_ was significantly shorter (MST: 49 vs. 757 days, = 0.0205). There were no significant differences between groups in the oral microbiota. This study revealed a strong association between gut microbiota diversity and ICI response in NSCLC patients. Moreover, specific gut microbiota compositions may influence the ICI response. These findings might be useful in identifying biomarkers to predict ICI response.