Overview of Dimensionality Reduction Techniques

As discussed in the Introduction section, the goal of any dimensionality reduction technique is to manage the sparsity of the dataset while keeping the useful information that is provided, so dimensionality reduction is typically an important pre-processing step used before a classification stage. Most dimensionality reduction techniques aim to complete this task using a process of feature projection, which adjusts the data from the higher dimensional space into a space with fewer dimensions to remove the sparsity from the data. Again, as a means of visualizing the projection process, consider a sphere in a 3D space. We can project the sphere into lower 2D space into a circle with some information loss (the value for the z coordinate) but retaining much of the information that describes its original shape. We still know the origin, radius, and manifold (outline) of the shape, and it is still very clear that it is a circle. So, if we were given...