▶Book Description
The big data architects are the "masters" of data, and hold high value in today's market. Handling big data, be it of good or bad quality, is not an easy task. The prime job for any big data architect is to build an end-to-end big data solution that integrates data from different sources and analyzes it to find useful, hidden insights.
Big Data Architect's Handbook takes you through developing a complete, end-to-end big data pipeline, which will lay the foundation for you and provide the necessary knowledge required to be an architect in big data. Right from understanding the design considerations to implementing a solid, efficient, and scalable data pipeline, this book walks you through all the essential aspects of big data. It also gives you an overview of how you can leverage the power of various big data tools such as Apache Hadoop and ElasticSearch in order to bring them together and build an efficient big data solution.
By the end of this book, you will be able to build your own design system which integrates, maintains, visualizes, and monitors your data. In addition, you will have a smooth design flow in each process, putting insights in action.
▶What You Will Learn
⦁ Learn Hadoop Ecosystem and Apache projects
⦁ Understand, compare NoSQL database and essential software architecture
⦁ Cloud infrastructure design considerations for big data
⦁ Explore application scenario of big data tools for daily activities
⦁ Learn to analyze and visualize results to uncover valuable insights
⦁ Build and run a big data application with sample code from end to end
⦁ Apply Machine Learning and AI to perform big data intelligence
⦁ Practice the daily activities performed by big data architects
▶Key Features
⦁ Learn to build and run a big data application with sample code
⦁ Explore examples to implement activities that a big data architect performs
⦁ Use Machine Learning and AI for structured and unstructured data
▶Who This Book Is For
Big Data Architect's Handbook is for you if you are an aspiring data professional, developer, or IT enthusiast who aims to be an all-round architect in big data. This book is your one-stop solution to enhance your knowledge and carry out easy to complex activities required to become a big data architect.
▶What this book covers
⦁ Chapter 1, Why Big Data?, explains what big data is, why we need big data, who should deal with big data, when to use big data, and how to use big data. The design consideration of the end-to-end big data solution, including cloud, Hadoop, network, analytics and so on, are also outlined here.
⦁ Chapter 2, Big Data Environment Setup, provides a step-by-step guide of how to setup environment to run big data applications.
⦁ Chapter 3, Hadoop Ecosystem, is about the Hadoop ecosystem. It consists of different open source modules, accessories, and Apache projects for reliable and scalable distributed computing. This chapter will teach you how to build a Hadoop big data system for streaming data with a step-by-step guide.
⦁ Chapter 4, NoSQL Database, explains the concepts, principles, properties, performance and hybrid of the popular NoSQL database so that a big data architect can confidently choose appropriate NoSQL for their projects. This chapter will teach you how to implement NoSQL for killer applications with a step-by-step guide.
⦁ Chapter 5, Off-the-Shelf Commercial Tools, introduces some popular commercial off-the-shelf tools for big data with a hands-on Stream Analytics example.
⦁ Chapter 6, Containerization, introduces the concept and application of container-based virtualization. It is an OS-level virtualization method for deploying and running distributed applications without launching an entire VM for each application. Moreover, management of Dockers and Kubernetes using Openshift is demonstrated here.
⦁ Chapter 7, Network Infrastructure, teaches essential network technology for an architect to design big data systems across racks, data centers, and geographical locations. Moreover, this chapter will teach you the network visualization tool via a step-by-step guide.
⦁ Chapter 8, Cloud Infrastructure, introduces essential considerations on cloud infrastructure design for big data from the perspective of performance and capability. The requirements of deploying big data in cloud are unique and quite different from traditional applications. Therefore, a big data architect must need careful design, especially estimating the amount of data to analyze by using the big data capability in the cloud, because not all public or private cloud offerings are built to accommodate big data solutions.
⦁ Chapter 9, Security and Monitoring, is about essential knowledge on security, including next-generation firewalls, DevOps security, and monitoring tools.
⦁ Chapter 10, Frontend Architecture, introduces the Frontend architecture, which is a collection of tools and processes that aims to improve the quality of our frontend code while creating a more efficient, scalable, and sustainable design for big data systems. To be a successful big data Architect, one critical factor is to present persuasive analytic results to mostly non-technical persons, such as C-level management, and decision-makers with a user-friendly, elegant, and responsive user graphic interface. This chapter will teach you how to use the React + Redux framework to build a responsive and easy debug user interface.
⦁ Chapter 11, Backend Architecture, shows how to design a scalable, resilient, manageable, and cost-effective distributed backend architecture with different combinations of technology. It handles business logic and data storage with a RESTful web API service.
⦁ Chapter 12, Machine Learning, teaches the essential concepts and killer applications of Machine Learning. You will learn about the most effective machine learning techniques, and gain practice implementing them and getting them to work for yourself and your enterprise. You'll learn about not only the theoretical underpinnings of learning, but also the practical know-how needed to quickly and powerfully apply these techniques to new problems.
⦁ Chapter 13, Artificial Intelligence, introduces AI and CNN with hands-on big data killer applications. The application for CNN or deep learning to work with machine learning is one good method to handle unstructured big data.
⦁ Chapter 14, Elasticsearch, shows how to use the open source tool Elasticsearch to do searching tasks in a big data system. This is because it is an enterprise-grade search engine, and easy to scale. More features of it are: handy REST API and JSON response, good documentation, Sense UI, stable and proven Lucene underlying engine, excellent Query DSL, multi-tenancy, advanced Search Features, configurable and extensible, percolation, custom analyzer, On-the-Fly Analyzer selection, rich ecosystem, and active community.
⦁ Chapter 15, Structured Data, introduces the use of open source tools to manipulate and analyze structured data.
⦁ Chapter 16, Unstructured Data, shows how to use open source tools to manipulate and analyze unstructured data. The readers will learn how to use machine learning and AI to extract information for analysis in killer applications such as a Retail Recommendation System and Facial Recognition.
⦁ Chapter 17, Data Visualization, illustrates how to use tools to present analytical results to users using two top-of-the-shelf tools, Matplotlib and D3.js.
⦁ Chapter 18, Financial Trading System, covers algorithmic trading benefits and strategies, and how to design and deploy an end-to-end Financial Trading System via a step-by-step guide.
⦁ Chapter 19, Retail Recommendation System, shows how to design and deploy an end-to-end Retail Recommendation System through a step-by-step guide.
