▶What You Will Learn
- Techniques to determine the validity and confidence level of data
- Apply quartiles and n-tiles to datasets to see how data is distributed into many buckets
- Create data pipelines that combine multiple data lifecycle steps
- Use built-in features to gain a deeper understanding of the data
- Apply Lasso regression analysis method to your data
- Compare Apache Spark API with traditional Apache Spark data analysis
▶Key Features
- A beginner's guide for performing data analysis loaded with numerous rich, practical examples
- Access to popular Scala libraries such as Breeze, Saddle for efficient data manipulation and exploratory analysis
- Develop applications in Scala for real-time analysis and machine learning in Apache Spark
▶Who This Book Is For
If you are a data scientist or a data analyst who wants to learn how to perform data analysis using Scala, this book is for you. All you need is knowledge of the basic fundamentals of Scala programming.
▶What this book covers
- Chapter 1, Scala Overview, gives you a quick run through Scala and its features. It will prepare you for upcoming chapters.
- Chapter 2, Data Analysis Life Cycle, turns the focus exclusively to data analysis and its typical life cycle. It provides an overview of the steps involved in the data analysis life cycle.
- Chapter 3, Data Ingestion, deep-dives into the data ingestion aspects of the data life cycle. It covers extraction, staging, validation, cleaning, and shaping data tasks. It highlights how to deal with the variety aspect of data, that is, how to handle data from different sources in different formats.
- Chapter 4, Data Exploration and Visualization, deep-dives into the data exploration and visualization parts of the life cycle. It familiarizes the reader with techniques for discovering inherent properties associated with data using statistical as well as visual methods.
- Chapter 5, Applying Statistics and Hypothesis Testing, provides an overview of the statistical methods used in data analysis and covers techniques for deriving meaningful insights from data.
- Chapter 6, Intro to Spark for Distributed Data Analysis, covers the transition to doing data analysis on distributed systems and doing it at scale. It provides a good introduction to Spark, a Scala-based distributed framework for data processing. It will guide you through Spark setup on your computer and introduce key features using practical examples.
- Chapter 7, Traditional Machine Learning for Data Analysis, covers topics such as decision trees, random forests, lasso regression, and k-means cluster analysis. It also covers the role of NLP in effectively analyzing certain types of data.
- Chapter 8, Near Real-Time Data Analysis Using Streaming, introduces the concept of streamoriented processing and compares it to traditional batch-oriented processing. It also illustrates how streaming can be used to perform near real-time data analysis. This chapter deep-dives into Spark Streaming and will guide you on implementing clustering and a classifier leveraging Spark Streaming APIs.
- Chapter 9, Working with Data at Scale, is dedicated to processing data at scale. It looks at data analysis from multiple dimensions, such as cost, reliability, and performance. It provides guidance on some of the best reliability and performance practices. It provides a complete picture of how a practical real-world data analysis life cycle works and will help you to put this into practice in a production environment.
