TensorFlow in Action

by Thushan Ganegedara

Overview

Unlock the TensorFlow design secrets behind successful deep learning applications! Deep learning StackOverflow contributor Thushan Ganegedara teaches you the new features of TensorFlow 2 in this hands-on guide.

In TensorFlow in Action you will learn:

    Fundamentals of TensorFlow
    Implementing deep learning networks
    Picking a high-level Keras API for model building with confidence
    Writing comprehensive end-to-end data pipelines
    Building models for computer vision and natural language processing
    Utilizing pretrained NLP models
    Recent algorithms including transformers, attention models, and ElMo

In TensorFlow in Action, you'll dig into the newest version of Google's amazing TensorFlow framework as you learn to create incredible deep learning applications. Author Thushan Ganegedara uses quirky stories, practical examples, and behind-the-scenes explanations to demystify concepts otherwise trapped in dense academic papers. As you dive into modern deep learning techniques like transformer and attention models, you’ll benefit from the unique insights of a top StackOverflow contributor for deep learning and NLP.

About the technology
Google’s TensorFlow framework sits at the heart of modern deep learning. Boasting practical features like multi-GPU support, network data visualization, and easy production pipelines using TensorFlow Extended (TFX), TensorFlow provides the most efficient path to professional AI applications. And the Keras library, fully integrated into TensorFlow 2, makes it a snap to build and train even complex models for vision, language, and more.

About the book
TensorFlow in Action teaches you to construct, train, and deploy deep learning models using TensorFlow 2. In this practical tutorial, you’ll build reusable skill hands-on as you create production-ready applications such as a French-to-English translator and a neural network that can write fiction. You’ll appreciate the in-depth explanations that go from DL basics to advanced applications in NLP, image processing, and MLOps, complete with important details that you’ll return to reference over and over.

What's inside

    Covers TensorFlow 2.9
    Recent algorithms including transformers, attention models, and ElMo
    Build on pretrained models
    Writing end-to-end data pipelines with TFX

About the reader
For Python programmers with basic deep learning skills.

About the author
Thushan Ganegedara is a senior ML engineer at Canva and TensorFlow expert. He holds a PhD in machine learning from the University of Sydney.

Table of Contents
PART 1 FOUNDATIONS OF TENSORFLOW 2 AND DEEP LEARNING
1 The amazing world of TensorFlow
2 TensorFlow 2
3 Keras and data retrieval in TensorFlow 2
4 Dipping toes in deep learning
5 State-of-the-art in deep learning: Transformers
PART 2 LOOK MA, NO HANDS! DEEP NETWORKS IN THE REAL WORLD
6 Teaching machines to see: Image classification with CNNs
7 Teaching machines to see better: Improving CNNs and making them confess
8 Telling things apart: Image segmentation
9 Natural language processing with TensorFlow: Sentiment analysis
10 Natural language processing with TensorFlow: Language modeling
PART 3 ADVANCED DEEP NETWORKS FOR COMPLEX PROBLEMS
11 Sequence-to-sequence learning: Part 1
12 Sequence-to-sequence learning: Part 2
13 Transformers
14 TensorBoard: Big brother of TensorFlow
15 TFX: MLOps and deploying models with TensorFlow

Product Details

ISBN-13:	9781638356738
Publisher:	Manning
Publication date:	11/01/2022
Sold by:	SIMON & SCHUSTER
Format:	eBook
Pages:	680
File size:	23 MB
Note:	This product may take a few minutes to download.

About the Author

Thushan Ganegedara is a data scientist with QBE. He holds a PhD in machine learning from the University of Sydney and he has worked with TensorFlow for almost 5 years. Thushan is also one of the most active answer providers for TensorFlow and TensorFlow2.0 tags on Stackoverflow, a DataCamp instructor, and has authored a book and video course on NLP with TensorFlow.

Table of Contents

Preface xiv

Acknowledgments xvi

About this book xvii

About the author xxi

About the cover illustration xxii

Part 1 Foundations of TensorFlow 2 and Deep Learning 1

1 The amazing world of TensorFlow 3

1.1 What is TensorFlow? 4

An overview of popular components of TensorFlow 6

Building and deploying a machine learning model 8

1.2 GPU vs. CPU 9

1.3 When and when not to use TensorFlow 10

When to use TensorFlow 10

When not to use TensorFlow 12

1.4 What will this book teach you? 14

TensorFlow fundamentals 14

Deep learning algorithms 14

Monitoring and optimization 14

1.5 Who is this book for? 15

1.6 Should we really care about Python and TensorFlow 2? 16

2 TensorFlow 2 19

2.1 First steps with TensorFlow 2 20

How does TensorFlow operate under the hood? 24

2.2 TensorFlow building blocks 28

Understanding tf. Variable 29

Understanding tf. Tensor 32

Understanding tf. Operation 35

2.3 Neural network-related computations in TensorFlow 39

Matrix multiplication 39

Convolution operation 41

Pooling operation 43

3 Keras and data retrieval in TensorFlow 2 47

3.1 Keras model-building APIs 48

Introducing the data set 49

The Sequential API 52

The functional API 56

The sub-classing API 61

3.2 Retrieving data for TensorFlow/Keras models 65

Tf.data API 66

Keras DataGenerators 72

Tensorflow-datasets package 75

4 Dipping toes in deep learning 80

4.1 Fully connected networks 81

Understanding the data 82

Autoencoder model 85

4.2 Convolutional neural networks 90

Understanding the data 90

Implementing the network 92

4.3 One step at a time: Recurrent neural networks (RNNs) 105

Understanding the data 107

Implementing the model 111

Predicting future CO2 values with the trained model 115

5 State-of-the-art in deep learning: Transformers 119

5.1 Representing text as numbers 120

5.2 Understanding the Transformer model 123

The encoder-decoder view of the Transformer 123

Diving deeper 124

Self-attention layer 128

Understanding self-attention using scalars 131

Self-attention as a cooking competition 135

Mashed self-attention layers 136

Multi-head attention 138

Fully connected layer 139

Putting everything together 141

Part 2 Look Ma, no hands! Deep networks in the real world 147

6 Teaching machines to see: Image classification with CNNs 149

6.1 Putting the data under the microscope: Exploratory data analysis 150

The folder/file structure 152

Understanding the classes in the data set 155

Computing simple statistics on the data set 158

6.2 Creating data pipelines using the Keras ImageDataGenerator 160

6.3 Inception net: Implementing a state-of-the-art image classifier 165

Recap on CNNs 166

Inception net v1 169

Putting everything together 181

Other Inception models 183

6.4 Training the model and evaluating performance 188

7 Teaching machines to see better: Improving CNNs and making them confess 194

7.1 Techniques for reducing overfitting 195

Image data augmentation with Keras 196

Dropout: Randomly switching off parts of your network to improve generalizability 203

Early stopping: Halting the training process if the network starts to underperform 207

7.2 Toward minimalism: Minception instead of Inception 210

Implementing the stem 211

Implementing Inception-ResNet type A block 216

Implementing the Inception-ResNet type B block 223

Implementing the reduction block 225

Putting everything together 227

Training Minception 229

7.3 If you can't beat them, join 'em: Using pretrained networks for enhancing performance 232

Transfer learning: Reusing existing knowledge in deep neural networks 232

7.4 Grad-CAM: Making CNNs confess 238

8 Telling things apart: Image segmentation 243

8.1 Understanding the data 245

8.2 Getting serious: Defining a TensorFlow data pipeline 251

Optimizing tf.data pipelines 263

The final tf.data pipeline 264

8.3 DeepLabv3: Using pretrained networks to segment images 266

A quick overview of the ResNet-50 model 268

Atrous convolution: Increasing the receptive field of convolution layers with holes 269

ImplementingDeepLab v3 using the Keras functional API 270

Implementing the atrous spatial pyramid pooling module 273

Putting everything together 275

8.4 Compiling the model: Loss functions and evaluation metrics in image segmentation 277

Loss functions 277

Evaluation metrics 283

8.5 Training the model 289

8.6 Evaluating the model 290

9 Natural language processhig with TensorFlow: Sentiment analysis 296

9.1 What the text? Exploring and processing text 298

9.2 Getting text ready for the model 308

Splitting training/validation and testing data 309

Analyze the vocabulary 313

Analyzing the sequence length 315

Text to words and then to numbers with Keras 316

9.3 Defining an end-to-end NLP pipeline with TensorFlow 319

9.4 Happy reviews mean happy customers: Sentiment analysis 325

LSTM Networks 326

Defining the final model 331

9.5 Training and evaluating the model 336

9.6 Injecting semantics with word vectors 339

Word embeddings 340

Defining the final model with word embeddings 341

Training and evaluating the model 344

10 Natural language processing with TensorFlow: Language modeling 349

10.1 Processing the data 350

What is language modeling? 351

Downloading and playing with data 351

Too large vocabulary? N-grams to the rescue 356

Tokenizing text 358

Defining a tf.data pipeline 360

10.2 GRUs in Wonderland: Generating text with deep learning 365

10.3 Measuring the quality of the generated text 370

10.4 Training and evaluating the language model 372

10.5 Generating new text from the language model: Greedy decoding 374

10.6 Beam search: Enhancing the predictive power of sequential models 379

Part 3 Advanced deep networks for complex problems 385

11 Sequence-to-sequence learning: Part 1 387

11.1 Understanding the machine translation data 388

11.2 Writing an English-German seq2seq machine translator 395

The TextVectorization layer 398

Defining the TextVectorization layers for the seq2seq model 400

Defining the encoder 401

Defining the decoder and the final model 405

Compiling the model 409

11.3 Training and evaluating the model 410

11.4 From training to inference: Defining the inference model 423

12 Sequence-to-sequence learning: Part 2 433

12.1 Eyeballing the past: Improving our model with attention 435

Implementing Bahdanau attention in TensorFlow 436

Defining the final model 440

Training the model 443

12.2 Visualizing the attention 445

13 Transformers 453

13.1 Transformers in more detail 455

Revisiting the basic components of the Transformer 455

Embeddings in the Transformer 457

Residuals and normalization 460

13.2 Using pretrained BERT for spam classification 463

Understanding BERT 465

Classifying spam with BERT in TensorFlow 470

13.3 Question answering with Hugging Face's Transformers 483

Understanding the data 485

Processing data 487

Defining the DistilBERT model 495

Training the model 502

Ask BERT a question 505

14 TensorBoard: Big brother of TensorFlow 511

14.1 Visualize data with TensorBoard 512

14.2 Tracking and monitoring models with TensorBoard 517

14.3 Using tf.summary to write custom metrics during model training 526

14.4 Profiling models to detect performance bottlenecks 529

Optimizing the input pipeline 536

Mixed precision training 539

14.5 Visualizing word vectors with the TensorBoard 544

15 TFX: MLOps and deploying models with TensorFlow 554

15.1 Writing a data pipeline with TFX 556

Loading data from CSV files 560

Generating basic statistics from the data 564

Inferring the schema from data 567

Converting data to features 569

15.2 Training a simple regression neural network: TFX Trainer API 576

Defining a Keras model 577

Defining the model training 584

SignatureDefs: Defining how models are used outside TensorFlow 586

Training the Keras model with TFX Trainer 589

15.3 Setting up Docker to serve a trained model 596

15.4 Deploying the model and serving it through an API 599

Validating the infrastructure 600

Resolving the correct model 602

Evaluating the model 602

Pushing the final model 608

Predicting with the TensorFlow serving API 608

Appendix A Setting up the environment 615

Appendix B Computer vision 625

Appendix C Natural language processing 639

Index 647