Filter-Based Feature Selection Using Information Theory and Binary Cuckoo Optimisation Algorithm

Dimensionality reduction is among the data mining process that is used to reduce the noise and complexity of features in various datasets. Feature selection (FS) is one of the most commonly used dimensionalities that reduces the unwanted features from the datasets. FS can be either wrapper or filter. Wrappers select subsets of the feature with better classification performance but are computationally expensive. On the other hand, filters are computationally fast but lack feature interaction among selected subsets of features which in turn affect the classification performance of the chosen subsets of features. This study proposes two concepts of information theory mutual information (MI). As well as entropy (E). Both were used together with binary cuckoo optimization algorithm BCOA (BCOA-MI and BCOA-EI). The target is to improve classification performance (reduce the error rate and computational complexity) on eight datasets with varying degrees of complexity. A support vector machine classifier was used to measure and computes the error rates of each of the datasets for both BCOA-MI and BCOA-E. The analysis of the results showed that BCOA-E selects a fewer number of features and performed better in terms of error rate. In contrast, BCOA-MI is computationally faster but chooses a larger number of features. Comparison with other methods found in the literature shows that the proposed BCOA-MI and BCOA-E performed better in terms of accuracy, the number of selected features, and execution time in most of the datasets.

Keywords:

Feature Selection , Filter-Based , Binary Cuckoo Optimization , information theory