To investigate potential uncertainties in CT-based non-small cell lung cancer (NSCLC) radiomics associated with feature selection methods, predictive models, and their related factors.. CT images from 496 pre-treatment NSCLC patients were retrospectively retrieved from a GE CT scanner. The original patient cohort (100%) was sampled to generate 25%, 50%, and 75% sub-cohorts to investigate potential impact of cohort size. Radiomic features were extracted from the lung nodule using IBEX. Five feature selection methods (analysis of variance, least absolute shrinkage and selection operator, mutual information, minimum redundancy-maximum relevance, Relief) and seven predictive models (DT-decision tree, RF-random forest, LR-logistic regression, SVC-support vector classifier, KNN-k-nearest neighbor, GB-gradient boost, NB-Naïve-Bayesian) were included for the analysis. Cohort size and cohort composition (i.e. same sized cohorts with partially different patients) were investigated as factors related to feature selection methods. The number of input features and model validation methods (2-, 5-, and 10-fold cross-validation) were investigated for predictive models. Using a two-year survival endpoint, AUC values were calculated for the various combinations.. Features ranked by different feature selection methods are not consistent and dependent on cohort size, even for the same methods. Two methods, Relief and LASSO, select 17 and 14 features from 25 common features to all cohort sizes, respectively, while other 3 feature selection methods have <10 features common to all cohort sizes. Feature rankings also highly depend on minor differences in cohort composition. AUCs for the 2100 tested combinations vary from 0.427 to 0.973. Among them, only 16 combinations achieve an AUC > 0.65. There is no clear path to reliable CT NSCLC radiomics.. The use of different feature selection methods and predictive models can generate inconsistent results. This should be further investigated to improve the reliability of radiomic studies.