Mastering Concurrency in Python

<Mastering Concurrency in Python> ▶Book Description
Python is one of the most popular programming languages, with numerous libraries and frameworks that facilitate high-performance computing. Concurrency and parallelism in Python are essential when it comes to multiprocessing and multithreading; they behave differently, but their common aim is to reduce the execution time. This book serves as a comprehensive introduction to various advanced concepts in concurrent engineering and programming.

Mastering Concurrency in Python starts by introducing the concepts and principles in concurrency, right from Amdahl's Law to multithreading programming, followed by elucidating multiprocessing programming, web scraping, and asynchronous I/O, together with common problems that engineers and programmers face in concurrent programming. Next, the book covers a number of advanced concepts in Python concurrency and how they interact with the Python ecosystem, including the Global Interpreter Lock (GIL). Finally, you'll learn how to solve real-world concurrency problems through examples.

By the end of the book, you will have gained extensive theoretical knowledge of concurrency and the ways in which concurrency is supported by the Python language

▶What You Will Learn
⦁ Explore the concepts of concurrency in programming
⦁ Explore the core syntax and features that enable concurrency in Python
⦁ Understand the correct way to implement concurrency
⦁ Abstract methods to keep the data consistent in your program
⦁ Analyze problems commonly faced in concurrent programming
⦁ Use application scaffolding to design highly-scalable programs

▶Key Features
⦁ Explore the core syntaxes, language features and modern patterns of concurrency in Python
⦁ Understand how to use concurrency to keep data consistent and applications responsive
⦁ Utilize application scaffolding to design highly-scalable programs

▶Who This Book Is For
This book is for developers who wish to build high-performance applications and learn about signle-core, multicore programming or distributed concurrency. Some experience with Python programming language is assumed.

▶What this book covers
⦁ Chapter 1, Advanced Introduction to Concurrent and Parallel Programming, introduces you to the concept of concurrency, and demonstrates an instance in which concurrent programming can improve significantly the speed of a Python program.

⦁ Chapter 2, Amdahl's Law, takes a theoretical approach and discusses the limitations of concurrency in improving the speed of applications. We will take a look at what concurrency truly provides and how we can best incorporate it.

⦁ Chapter 3, Working with Threads in Python, introduces the formal definition of threading and covers a different approach to implementing threading in a Python program. In this chapter, we will also discuss a major element in concurrent programming—the concept of synchronization.

⦁ Chapter 4, Using the with Statement in Threads, combines the concept of context management with threading in the overall context of concurrent programming in Python. We will be introduced to the main idea behind context management and how it is used in various programming practices, including threading.

⦁ Chapter 5, Concurrent Web Requests, covers one of the main applications of concurrent programming: web scraping. It also covers the concept of web scraping, along with other relevant elements, before discussing how threading can be applied to web scraping programs in order to achieve significant speedup.

⦁ Chapter 6, Working with Processes in Python, shows the formal definition of multiprocessing and how Python supports it. We will also learn more about the key differences between threading and multiprocessing, which are often confused with one another.

⦁ Chapter 7, Reduction Operators in Processes, pairs the concepts of reduction operations and multiprocessing together as a concurrent programming practice. This chapter will go over the theoretical foundation of reduction operations and how it is relevant to multiprocessing as well as programming in general.

⦁ Chapter 8, Concurrent Image Processing, goes into a specific application of concurrency: image processing. The basic ideas behind image processing, in addition to some of the most common processing techniques, are discussed. We will, of course, see how concurrency, specifically multiprocessing, can speed up the task of image processing.

⦁ Chapter 9, Introduction to Asynchronous Programming, considers the formal concept of asynchronous programming as one of the three major concurrent programming models aside from threading and multiprocessing. We will learn how asynchronous programming is fundamentally different from the two mentioned, but can still speedup concurrent applications.

⦁ Chapter 10, Implementing Asynchronous Programming in Python, goes in depth into the API that Python provides to facilitate asynchronous programming. Specifically, we will learn about the asyncio module, which is the main tool for implementing asynchronous programming in Python, and the general structure of an asynchronous application.

⦁ Chapter 11, Building Communication Channels with asyncio, combines the knowledge obtained regarding asynchronous programming covered in previous chapters with the topic of network communication. Specifically, we will look into using the aiohttp module as a tool to make asynchronous HTTP requests to web servers, as well as the aiofile module that implements asynchronous file reading/writing.

⦁ Chapter 12, Deadlocks, introduces the first of the problems that are commonly faced in concurrent programming. We will learn about the classical dining philosophers problem as an example of how deadlocks can cause concurrent programs to stop functioning. This chapter will also cover a number of potential approaches to deadlocks as well as relevant concepts, such as livelocks and distributed deadlocks.

⦁ Chapter 13, Starvation, considers another common problem in concurrent applications. The chapter uses the narrative of the classical readers-writers problem to explain the concept of starvation and its causes. We will, of course, also discuss potential solutions to these problems via hands-on examples in Python.

⦁ Chapter 14, Race Conditions, addresses arguably the most well-known concurrency problem: race conditions. We will also discuss the concept of a critical section, which is an essential element in the context of race conditions specifically, and concurrent programming in general. The chapter will then cover mutual exclusion as a potential solution for this problem.

⦁ Chapter 15, The Global Interpreter Lock, introduces the infamous GIL, which is considered the biggest challenge in concurrent programming in Python. We will learn about the reason behind GIL's implementation and the problems that it raises. This chapter concludes with some thoughts regarding how Python programmers and developers should think about and interact with the GIL.

⦁ Chapter 16, Designing Lock-Based and Mutex-Free Concurrent Data Structures, analyzes the process of designing two common concurrent data structures involving locks as a synchronization mechanism: lock-based and mutex-free. Several advanced analyses of the implementation of the data structures, as well as the performance thereof, are incorporated into the chapter so that readers will develop a critical mindset when it comes to designing concurrent applications.

⦁ Chapter 17, Memory Models and Operations on Atomic Types, includes theoretical topics that involve the underlying structure of the Python language and how programmers can take advantage of that in their concurrent applications. The concept of atomic operations is also introduced to readers in this chapter.

⦁ Chapter 18, Building a Server from Scratch, walks readers through the process of building a non-blocking server on a low level. We will learn about network programming functionalities that the socket module in Python provides and how we can use them to implement a functioning server. We will also apply the general structure of an asynchronous program discussed earlier in the book to convert a blocking server into a non-blocking one.

⦁ Chapter 19, Testing, Debugging, and Scheduling Concurrent Applications, covers higher-level uses of concurrent programs. The chapter will first cover how concurrency can be applied to the task of scheduling Python applications via the APScheduler module. We will then discuss the complexities that arise from concurrency in the topics of testing and debugging Python programs.