Getting Started with Google BERT

Kickstart your NLP journey by exploring BERT and its variants such as ALBERT, RoBERTa, DistilBERT, VideoBERT, and more with Hugging Face's transformers library

▶What You Will Learn
-Understand the transformer model from the ground up
-Find out how BERT works and pre-train it using masked language model (MLM) and next sentence prediction (NSP) tasks
-Get hands-on with BERT by learning to generate contextual word and sentence embeddings
-Fine-tune BERT for downstream tasks
-Get to grips with ALBERT, RoBERTa, ELECTRA, and SpanBERT models
-Get the hang of the BERT models based on knowledge distillation
-Understand cross-lingual models such as XLM and XLM-R
-Explore Sentence-BERT, VideoBERT, and BART

▶Key Features
-Explore the encoder and decoder of the transformer model
-Become well-versed with BERT along with ALBERT, RoBERTa, and DistilBERT
-Discover how to pre-train and fine-tune BERT models for several NLP tasks

▶Who This Book Is For
This book is for NLP professionals and data scientists looking to simplify NLP tasks to enable efficient language understanding using BERT. A basic understanding of NLP concepts and deep learning is required to get the best out of this book.

▶What this book covers
- Chapter 1, A Primer on Transformers, explains the transformer model in detail. We will understand how the encoder and decoder of transformer work by looking at their components in detail.

- Chapter 2, Understanding the BERT model, helps us to understand the BERT model. We will learn how the BERT model is pre-trained using Masked Language Model (MLM) and Next Sentence Prediction (NSP) tasks. We will also learn several interesting subword tokenization algorithms.

- Chapter 3, Getting Hands-On with BERT, explains how to use the pre-trained BERT model. We will learn how to extract contextual sentences and word embeddings using the pretrained BERT model. We will also learn how to fine-tune the pre-trained BERT for downstream tasks such as question-answering, text classification, and more.

- Chapter 4, BERT Variants I – ALBERT, RoBERTa, ELECTRA, and SpanBERT, explains several variants of BERT. We will learn how BERT variants differ from BERT and how they are useful in detail.

- Chapter 5, BERT Variants II – Based on Knowledge Distillation, deals with BERT models based on distillation, such as DistilBERT and TinyBERT. We will also learn how to transfer knowledge from a pre-trained BERT model to a simple neural network.

- Chapter 6, Exploring BERTSUM for Text Summarization, explains how to fine-tune the pretrained BERT model for a text summarization task. We will understand how to fine-tune BERT for extractive summarization and abstractive summarization in detail.

- Chapter 7, Applying BERT to Other Languages, deals with applying BERT to languages other than English. We will learn about the effectiveness of multilingual BERT in detail. We will also explore several cross-lingual models such as XLM and XLM-R.

- Chapter 8, Exploring Sentence and Domain-Specific BERT, explains Sentence-BERT, which is used to obtain the sentence representation. We will also learn how to use the pre-trained Sentence-BERT model. Along with this, we will also explore domain-specific BERT models such as ClinicalBERT and BioBERT.

- Chapter 9, Working with VideoBERT, BART, and More, deals with an interesting type of BERT called VideoBERT. We will also learn about a model called BART in detail. We will also explore two popular libraries known as ktrain and bert-as-service.