Dimensionality Reduction and Feature Selection in Big Data using Improved Pso and New Clustering Techniques

Data is usually described by a large number of attributes. Many of these features may be irrelevant and redundant for the intended data mining application. The presence of many of these unrelated features in the data set negatively affects the performance of the machine learning algorithm and also increases the computational complexity, so reducing the size of the data set is a key task in data mining and machine learning applications. In this paper, we try to present a new model of particle optimization algorithms that have new operators to improve their search capability, as well as a new feature clustering algorithm, a new feature selection method using the node centrality criterion. In this proposed method, in most of the data sets used, the new algorithm has less execution time and more classification accuracy or is comparable to other feature selection methods.

Keywords:

feature selection , big data , parallel processing , map-reduce model , Hadoop platform , particle optimization algorithm , graph clustering