Getting Started with Storm: Continuous Streaming Computation with Twitter's Cluster Technology

by Jonathan Leibiusky, Gabriel Eisbruch, Dario Simonassi

Overview

Even as big data is turning the world upside down, the next phase of the revolution is already taking shape: real-time data analysis. This hands-on guide introduces you to Storm, a distributed, JVM-based system for processing streaming data. Through simple tutorials, sample Java code, and a complete real-world scenario, you’ll learn how to build fast, fault-tolerant solutions that process results as soon as the data arrives.

Discover how easy it is to set up Storm clusters for solving various problems, including continuous data computation, distributed remote procedure calls, and data stream processing.

• Learn how to program Storm components: spouts for data input and bolts for data transformation
• Discover how data is exchanged between spouts and bolts in a Storm topology
• Make spouts fault-tolerant with several commonly used design strategies
• Explore bolts—their life cycle, strategies for design, and ways to implement them
• Scale your solution by defining each component’s level of parallelism
• Study a real-time web analytics system built with Node.js, a Redis server, and a Storm topology
• Write spouts and bolts with non-JVM languages such as Python, Ruby, and Javascript

Product Details

ISBN-13:	9781449324049
Publisher:	O'Reilly Media, Incorporated
Publication date:	08/31/2012
Sold by:	Barnes & Noble
Format:	eBook
Pages:	106
File size:	4 MB

About the Author

Jonathan Leibiusky, Head of Research and Development at MercadoLibre, has been working in software development for more than 10 years. He has developed and contributed to several new and existing open source projects, including "Jedis", which is being used actively by VMWare and SpringSource.

Gabriel is a computer science student and works as an Software Architect in Mercadolibre (NASDAQ MELI) since 2007. He is tasked with researching technologies and developing projects. In the last year he has specialized in big data analysis, implementing Mercadolibre's hadoop cluster.

Dario has been working in software development for more than 10 years. Since 2004 he has specialized in large website, operations and performance. Today, Dario is the Chief Architect of MercadoLibre (NASDAQ MELI) where he leads the architecture team.

Table of Contents

Preface; Conventions Used in This Book; Using Code Examples; Safari® Books Online; How to Contact Us; Acknowledgements; Chapter 1: Basics; 1.1 The Components of Storm; 1.2 The Properties of Storm; Chapter 2: Getting Started; 2.1 Operation Modes; 2.2 Hello World Storm; 2.3 Creating Our First Topology; 2.4 Conclusion; Chapter 3: Topologies; 3.1 Stream Grouping; 3.2 LocalCluster versus StormSubmitter; 3.3 DRPC Topologies; Chapter 4: Spouts; 4.1 Reliable versus Unreliable Messages; 4.2 Getting Data; 4.3 Conclusion; Chapter 5: Bolts; 5.1 Bolt Lifecycle; 5.2 Bolt Structure; 5.3 Reliable versus Unreliable Bolts; 5.4 Multiple Streams; 5.5 Multiple Anchoring; 5.6 Using IBasicBolt to Ack Automatically; Chapter 6: A Real-Life Example; 6.1 The Node.js Web Application; 6.2 Starting the Node.js Web Application; 6.3 The Storm Topology; 6.4 The Redis Server; 6.5 Testing the Topology; 6.6 Notes on Scalability and Availability; Chapter 7: Using Non-JVM Languages with Storm; 7.1 The Multilang Protocol Specification; Chapter 8: Transactional Topologies; 8.1 The Design; 8.2 Transactions in Action; 8.3 Partitioned Transactional Spouts; 8.4 Opaque Transactional Topologies; Installing the Storm Client; Installing Storm Cluster; Real Life Example Setup; Installing Redis; Installing Node.js; Building and Testing; Running the Topology; Playing with the Example;