Overview of Dimensionality Reduction Techniques

The goal of any dimensionality reduction technique is to manage the sparsity of the dataset while keeping any useful information that is provided. In our case of classification, dimensionality reduction is typically used as an important preprocessing step used before the actual classification. Most dimensionality reduction techniques aim to complete this task using a process of feature projection, which adjusts the data from the higher-dimensional space into a space with fewer dimensions to remove the sparsity from the data. Again, as a means of visualizing the projection process, consider a sphere in a 3D space. We can project the sphere into a lower 2D space into a circle with some information loss (the value for the z coordinate), but retaining much of the information that describes its original shape. We still know the origin, radius, and manifold (outline) of the shape, and it is still very clear that it is a circle. So, depending...