Chapter 1, Why Big Data?, explains what big data is, why we need big data, who should deal with big data, when to use big data, and how to use big data. The design consideration of the end-to-end big data solution, including cloud, Hadoop, network, analytics and so on, are also outlined here.
Chapter 2, Big Data Environment Setup, provides a step-by-step guide of how to setup environment to run big data applications.
Chapter 3, Hadoop Ecosystem, is about the Hadoop ecosystem. It consists of different open source modules, accessories, and Apache projects for reliable and scalable distributed computing. This chapter will teach you how to build a Hadoop big data system for streaming data with a step-by-step guide.
Chapter 4, NoSQL Database, explains the concepts, principles, properties, performance and hybrid of the popular NoSQL database so that a big data architect can confidently choose appropriate NoSQL for their projects. This chapter will teach you how to implement NoSQL for killer applications with a step-by-step guide.
Chapter 5, Off-the-Shelf Commercial Tools, introduces some popular commercial off-the-shelf tools for big data with a hands-on Stream Analytics example.
Chapter 6, Containerization, introduces the concept and application of container-based virtualization. It is an OS-level virtualization method for deploying and running distributed applications without launching an entire VM for each application. Moreover, management of Dockers and Kubernetes using Openshift is demonstrated here.
Chapter 7, Network Infrastructure, teaches essential network technology for an architect to design big data systems across racks, data centers, and geographical locations. Moreover, this chapter will teach you the network visualization tool via a step-by-step guide.
Chapter 8, Cloud Infrastructure, introduces essential considerations on cloud infrastructure design for big data from the perspective of performance and capability. The requirements of deploying big data in cloud are unique and quite different from traditional applications. Therefore, a big data architect must need careful design, especially estimating the amount of data to analyze by using the big data capability in the cloud, because not all public or private cloud offerings are built to accommodate big data solutions.
Chapter 9, Security and Monitoring, is about essential knowledge on security, including next-generation firewalls, DevOps security, and monitoring tools.
Chapter 10, Frontend Architecture, introduces the Frontend architecture, which is a collection of tools and processes that aims to improve the quality of our frontend code while creating a more efficient, scalable, and sustainable design for big data systems. To be a successful big data Architect, one critical factor is to present persuasive analytic results to mostly non-technical persons, such as C-level management, and decision-makers with a user-friendly, elegant, and responsive user graphic interface. This chapter will teach you how to use the React + Redux framework to build a responsive and easy debug user interface.
Chapter 11, Backend Architecture, shows how to design a scalable, resilient, manageable, and cost-effective distributed backend architecture with different combinations of technology. It handles business logic and data storage with a RESTful web API service.
Chapter 12, Machine Learning, teaches the essential concepts and killer applications of Machine Learning. You will learn about the most effective machine learning techniques, and gain practice implementing them and getting them to work for yourself and your enterprise. You'll learn about not only the theoretical underpinnings of learning, but also the practical know-how needed to quickly and powerfully apply these techniques to new problems.
Chapter 13, Artificial Intelligence, introduces AI and CNN with hands-on big data killer applications. The application for CNN or deep learning to work with machine learning is one good method to handle unstructured big data.
Chapter 14, Elasticsearch, shows how to use the open source tool Elasticsearch to do searching tasks in a big data system. This is because it is an enterprise-grade search engine, and easy to scale. More features of it are: handy REST API and JSON response, good documentation, Sense UI, stable and proven Lucene underlying engine, excellent Query DSL, multi-tenancy, advanced Search Features, configurable and extensible, percolation, custom analyzer, On-the-Fly Analyzer selection, rich ecosystem, and active community.
Chapter 15, Structured Data, introduces the use of open source tools to manipulate and analyze structured data.
Chapter 16, Unstructured Data, shows how to use open source tools to manipulate and analyze unstructured data. The readers will learn how to use machine learning and AI to extract information for analysis in killer applications such as a Retail Recommendation System and Facial Recognition.
Chapter 17, Data Visualization, illustrates how to use tools to present analytical results to users using two top-of-the-shelf tools, Matplotlib and D3.js.
Chapter 18, Financial Trading System, covers algorithmic trading benefits and strategies, and how to design and deploy an end-to-end Financial Trading System via a step-by-step guide.
Chapter 19, Retail Recommendation System, shows how to design and deploy an end-to-end Retail Recommendation System through a step-by-step guide.
