Big data management guarantees a certain level of data quality and accessibility for business intelligence, and also for big data analytics applications. Big data management incorporates the policies, techniques, and processes used for data collection, storage, administration, and the delivery of large repositories. The step can involve cleaning, integration, migration, and reporting.

Data ingestion refers to the process of procuring data into the system. This can be done via manual, semi-automatic, or automatic methods.

In a smaller system, users prefer to have some kind of web form...

There are different varieties of big data management solution that organizations can choose from. Different vendors support different technological stacks, and have different pricing models. There are vendors that offer a variety of standalone or multi-featured big data management tools. In this section, we will present some services from data management vendors.

Data cleansing is the process of identifying and fixing corrupt or fallacious records in a record set, table, or database. It also deals with identifying incomplete, incorrect, inaccurate, or irrelevant parts of the data, and then replacing, modifying, or deleting the infected data. Data entry and acquisition is inherently...

Due to the high prevalence of big data and its use in business intelligence, there are thousands of big data management vendors available in the market, ready to provide all data management services. Going through each of them would be beyond the scope of this book. In this section, we will explore ten big data management vendors. The list contained in this book does not guarantee the popularity or performance of each vendor. The list was created based on the use of state-of-the-art data analysis:

• Alation: Alation is a newer organization, providing cloud-based data services that were created to centralize knowledge about a company's data and how to apply it. You can find out more about the company from the link provided in the Further study section.
• AtScale: AtScale provides technology to facilitate familiar BI tools, in order to query data stored...

We have now explored what big data is, as well as its importance. A demand for storing and processing massive-scale datasets has encouraged the development of data storage and database systems over the past few years. With this advancement, storage systems and data models for big data have been improved and enhanced over time, from local storage to clustered, distributed to cloud-based storage. In this section, you will learn about some most popular storage models and data models:

The preceding diagram shows the taxonomy of data stores and data platforms, and the corresponding systems that are currently available in the market. Let's dig into each category in the next section.

Big data programming models portray the style of and show the interface paradigm for developers to write big data applications and programs. Programming models are normally the core feature of big data frameworks, as they implicitly affect the execution model of big data processing engines, and also drive revealing and constructing big data applications and programs.

In this section, we will discuss and compare the major programming models for writing big data applications, based on their taxonomy.

MapReduce refers to a programming model that is suitable for processing big amounts of data. For example, Hadoop is capable of executing a MapReduce program written in several programming languages...

In this section, we will get started with the most common open source programming languages in the big data world. We will show you how to get started with Python and R on a Windows machine and a macOS machine.

macOS X, High Sierra has a preloaded version of Python 2.7 out-of-the-box. If you have macOS X, you will not have to install or configure anything else in order to use Python 2. If you want to use Python3, then follow these instructions.

Before you install Python, you’ll be required to install GCC:

• GCC can be obtained by downloading XCode (https://developer.apple.com/xcode/).
• The smaller command-line tools, or the even smaller OSX-GCC-Installer package, can...

In this chapter, we discussed various theoretical aspects of big data management. Additionally, we enlightened you about different data management services, including data cleansing and data integration. There are also different storage models and data models, which we covered briefly in this chapter. Finally, in this chapter, we explained the different programming models, and how to get started with the Python and R programming languages. In the next chapter, you will learn more about the different types of data models, and the structured, unstructured, and operations performed on them.

• https://alation.com/
• http://atscale.com/
• https://www.cloudera.com/
• https://www.collibra.com/
• https://www.verato.com/
• https://hortonworks.com/
• https://www.sas.com/en_us/software/master-data-management.html
• https://www.verato.com/
• https://www.tibco.com/
• https://www.talend.com/
• https://www.trifacta.com/
• EBS Amazon. Elastic block store has launched all things distributed (2008). https://aws.amazon.com/ebs/
• OpenStack. OpenStack block storage cinder. https://wiki.openstack.org/wiki/Cinder. Accessed 27 March 2018
• K. Shvachko, H. Kuang, S. Radia, R. Chansler, The Hadoop distributed file system. in IEEE MSST (2010)
• https://aws.amazon.com/rds/
• https://azure.microsoft.com/en-us/overview/what-is-azure/
• https://wordpress.org
• https://cloud.google.com/sql/docs/

• https://www.mongodb.com
• http://couchdb.apache.org/
• https://www.python.org/downloads/windows/