Building a distributed pipeline is a huge — and complex — undertaking. If you want to ensure yours is scalable, has fast in-memory processing, can handle real-time or streaming data feeds with high throughput and low-latency, is well suited for ad-hoc queries, can be spread across multiple data centers, is built to allocate resources efficiently, and is designed to allow for future changes, join Andy Petrella from Data Fellas for this immensely practical hands-on course.

• Introduction
• Data Collection
• Akka Collectors
• Queuing
• Kafka

Break

• Interactive Programming
• Spark Notebook
• Produce (Notebook example)
• Live Coding Exercise

Break

• Streaming Data
• Spark Streaming
• Consume Streaming
• In-Memory Data
• Cassandra

Lunch

• Store
• Data Analysis
• Spark Core and MLlib
• Model (Notebook example)
• Live Coding Exercise

Break

• Access Layer
• Akka Micro Service
• Serve (Notebook example)

Break

• Cluster Manager and Orchestration
• Mesos
• Marathon and Chronos
• Live Coding Exercise
• Wrap Up

Instructor

Andy Petrella

Andy is a mathematician turned into a distributed computing entrepreneur. Besides being a Scala/Spark trainer. Andy also participated in many projects built using spark, cassandra, and other distributed technologies, in various fields including Geospatial, IoT, Automotive and Smart cities projects. He is the creator of the Spark Notebook, the only reactive and fully Scala notebook for Apache Spark.

In 2015, Xavier Tordoir and Andy founded Data Fellas around their product Agile Data Science Toolkit which facilitates the productization of Data Science projects and guarantees their maintainability and sustainability in time. Andy is also member of program committee of the O’Reilly Strata, Scala eXchange, Data Science eXchange and Devoxx events.

References

Andy Petrella presents Agile Data Science with Scala at SF Scala