A novel interval sparse evolutionary algorithm for efficient spectral variable selection.

In spectral analysis, selecting the right spectral variables is crucial for effective modeling. It reduces data dimensionality, removes irrelevant wavelength points, and improves both the generalization ability and computational efficiency of the model. However, the number of available samples often falls short of the total possible combinations of wavelengths, making variable selection a non-deterministic polynomial-time (NP) hard optimization problem. The current dedicated variable selection and heuristic algorithms fail to balance the effectiveness and speed of variable selection. Therefore, there is a great need for a more advanced approach to address this problem. (92) RESULTS: In this paper, we adopt a different perspective by considering variable selection as a large-scale sparse multi-objective optimization problem, modeled with fewer variables to achieve lower prediction errors. Then a novel interval sparse evolutionary algorithm (ISEA) was proposed, merging the benefits of dedicated variable selection algorithms and evolutionary algorithms. It incorporates a roulette probability mechanism and enhances the selection probability of key informative variables through a sparse population initialization strategy (SPIS) and a regional sparse evolution strategy (RSES). Specifically, the SPIS prioritizes variable regions through interval partial least squares (iPLS) and initializes the sparse population based on regional roulette probability, thereby enhancing the likelihood of selection of important regional variables in the initial sparse population. The RSES further focuses on more important regions, ensuring the variables in more important regions have a higher survival probability in subsequent generations. (138) SIGNIFICANCE: Applied to datasets of corn oil, soil, and diesel fuels, ISEA outperforms nine state-of-the-art methods by maintaining both the effectiveness of variable selection and running speed. Additionally, unlike dedicated variable selection algorithms, it is suitable for both specific variable selection scenarios and other large-scale sparse problems, such as critical node detection, frequent pattern mining, and neural network node training, demonstrating wide application potential. (63).