Voice Applications for Alexa and Google Assistant

by Dustin A. Coates

Overview

Summary

Voice Applications for Alexa and Google Assistant is your guide to designing, building, and implementing voice-based applications for Alexa and Google Assistant. Inside, you'll learn how to build your own "skills"—the voice app term for actions the device can perform—from scratch.

Foreword by Max Amordeluso.

Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. You'll find registration instructions inside the print book.

About the Technology

In 2018, an estimated 100 million voice-controlled devices were installed in homes worldwide, and the apps that control them, like Amazon Alexa and Google Assistant, are getting more powerful, with new skills being added every day. Great voice apps improve how users interact with the web, whether they're checking the weather, asking for sports scores, or playing a game.

About the Book

Voice Applications for Alexa and Google Assistant is your guide to designing, building, and implementing voice-based applications for Alexa and Google Assistant. You'll learn to build applications that listen to users, store information, and rely on user context, as you create a voice-powered sleep tracker from scratch. With the basics mastered, you'll dig deeper into multiuse conversational flow and other more-advanced concepts. Smaller projects along the way reinforce your new techniques and best practices.

What's inside

• Building a call-and-response skill
• Designing a voice user interface
• Using conversational context
• Going multimodal
• Tips and best practices

About the Reader

Perfect for developers with intermediate JavaScript skills and basic Node.js skills. No previous experience with voice-first platforms is required.

About the Author

Dustin A. Coates is a developer who focuses on voice and conversational applications. He's currently the voice search lead at Algolia and is also a Google Developers Expert for Assistant as well as cohost of the VUX World podcast.

Table of Contents

1. Introduction to voice first
2. Building a call-and-response skill on Alexa
3. Designing a voice user interface
4. Using entity resolution and built—in intents in Alexa skills
5. Making a conversational Alexa skill
6. VUI and conversation best practices
7. Using conversation tools to add meaning and usability
8. Directing conversation flow
9. Building for Google Assistant
10. Going multimodal
11. Push interactions
12. Building for actions on Google with the Actions SDK

Product Details

ISBN-13:	9781617295317
Publisher:	Manning
Publication date:	07/18/2019
Edition description:	1st Edition
Pages:	264
Product dimensions:	7.30(w) x 9.10(h) x 0.70(d)

About the Author

Dustin A. Coates is a developer who focuses on voice and conversational applications. He's currently the voice search lead at Algolia and is also a Google Developers Expert for Assistant as well as cohost of the VUX World podcast.

Table of Contents

Foreword xi

Preface xiii

Acknowledgments xv

About this book xvi

1 Introduction to voice first 1

1.1 What is voice first? 2

1.2 Designing for voice UIs 4

1.3 Anatomy of a voice command 5

Waking the voice-first device 7

Introducing natural language processing 8

How speech becomes text 8

Intents are the functions of a skill 10

Training the NLU with sample utterances 10

Plucking pertinent information from spoken text 12

1.4 The fulfillment code that ties it all together 13

1.5 Telling the device what to say 13

2 Building a call-and-response skill on Alexa 15

2.1 Skill metadata 16

Interaction model 18

Invocation name 19

Intents 20

Sample utterances 20

Slots 22

2.2 The interaction model 25

Building the intent 25

Slots 27

2.3 Fulfillment 28

Hosted endpoint 29

AWS Lambda 29

Coding the fulfillment 30

3 Designing a voice user interface 38

3.1 VUI fundamentals 39

3.2 The cooperative principle 40

Quantity 41

Quality 42

Relation 43

Manner 44

3.3 VUI planning 45

3.4 Variety 46

4 Using entity resolution and built-in intents in Alexa skills 49

4.1 Alexa Skills Kit CLI 50

Creating an Alexa skill project 52

4.2 Entity resolution 54

Fulfillment 57

Built-in intents 60

LaunchRequest 63

4.3 Invoking the skill locally 64

5 Making a conversational Alexa skill 67

5.1 Creating a conversation 68

State management 68

Per-state handlers 73

Handling the unhandled 76

5.2 Maintaining long-term information 77

5.3 Putting it all together 82

New intents 82

New utterances 82

New fulfillment 83

Correcting a mistake 89

6 VUI and conversation best practices 91

6.1 Conversations and context 91

6.2 A skill with context 93

Frame-based interactions 94

The fulfillment 96

Decaying context 100

6.3 Intercepting responses and requests 104

6.4 Unit testing 108

7 Using conversation tools to add meaning and usability 111

7.1 Discourse markers 111

7.2 Controlling the application's speech with SSML 115

Breaks and pauses 116

Prosody 118

amazan:effect 121

w, say-as 122

Phoneme 124

7.3 Embedding audio 125

8 Directing conversation flow 128

8.1 Guiding user interaction 129

8.2 Dialog interface 129

Creating the skill 133

Setting up the dialog model 133

Slot filling 135

Intent confirmation 141

Dialog model fulfillment 144

8.3 Handling errors 151

9 Building for Google Assistant 154

9.1 Setting up the application 155

9.2 Building the interaction model 156

Building an intent 159

Testing with the simulator 161

Parameters and entities 162

Adding entities 163

Using parameters in intents 164

9.3 Fulfillment 165

The code 167

Deployment 168

Changing the invocation name 169

10 Going multimodal 171

10.1 Introducing multimodal 172

10.2 Multimodal in actions 173

Simple responses 173

Rich responses 175

List responses 177

Suggestion chips 181

10.3 Surface capabilities 182

10.4 Multisurface conversations 183

11 Push interactions 187

11.1 Routine suggestions 188

Storing user data 189

Action suggestion for a routine 192

11.2 Daily updates 193

Develaper control of daily updates 193

11.3 Push notifications 197

11.4 Implicit invocation 201

12 Building for actions on Google with the Actions SDK 204

12.1 Dialogflow and the Actions SDK 204

12.2 App planning 206

12.3 The action package 207

12.4 The fulfillment 210

Parsing input with regular expressions 212

Handling the unexpected 220

Appendix A Adding an AWS IAM profile 222

Appendix B Connecting DynamoDB to a Lambda function 229

Glossary 231

Index 233