Parallel Processing in R

In data science application development, such as credit card fraud detection, airline delay prediction, sentiment analysis from a huge corpus of text, and so on, we are required to store, process, and analyze a dataset that might not fit into computer memory. Moreover, in some situations, the dataset might not be that big but the complexity of the algorithm forces us to use huge memory. In these types of situations where the dataset is way too big, or the algorithm is too complex, you are required to use parallel processing to achieve the task. In R, the data frame is the most convenient and popular structure to store, process, and analyze a dataset, but for a larger data context, the data frame is not fast enough. The external data frame (XDF) is an alternative to the typical R data frame used to store, process, and analyze larger data. In this chapter...