Optimized Dimensionality Reduction Using Metaheuristic and Class Separability

The high dimensionality of modern datasets presents significant challenges for machine learning, including increased computational cost, model complexity, and risk of overfitting. This study introduces a metaheuristic framework for optimized dimensionality reduction to identify the highly discriminative feature subsets. The proposed method (KDR-PSO) combines a Particle Swarm Optimization (PSO) algorithm with the K-Nearest Neighbors Distance Ratio (KDR) as a filter-based objective function. This metric quantitatively assesses class separability within a feature subspace by computing the ratio of the average distance from a sample to neighbors in other classes versus those in its own class. Maximizing this ratio with a penalty for model size, KDR-PSO automates the discovery of parsimonious feature sets that maximize inter-class discrimination. The method is computationally efficient, naturally lending itself to multi-class classification and avoiding the prohibitive cost associated with classifier-in-the-loop wrappers. Experimental results on benchmark gene expression and image datasets show that KDR-PSO can achieve better dimensionality reduction compared to baselines and other algorithms, such as winning a better or at least similar performing models with decreased features. This approach is a strong and pragmatic technique to improve the model interpretability and generalizability for high-dimensional regions.