addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo
This Meetup is cancelled

Introduction to Spark Structured Streaming

  • IBM Client Center

    1 Rogers Street, Cambridge, MA (map)

    42.366467 -71.077354

  • I’m excited to announce some additions to the Meetup agenda.

    First, we’ll be joined by Jonathan Gray, CEO of Cask. Prior to founding Cask, Jon was a software engineer at Facebook, where he was responsible for HBase engineering efforts, including Facebook Messenger and several other large-scale projects. Jon is a core contributor and active committer in the Big Data community. Jon will give a short talk on the evolution of the Big Data ecosystem and the projects he’s worked on as well as give us his perspective on what lies ahead for Hadoop, Spark and Big Data.

    Second, Scott Nichols, a Boston-based singer songwriter who plays regularly in clubs and venues in Boston and the New England area, will be performing live prior to the start of the technical content. http://www.scottnicholsmusic.com/

    Here is the revised agenda for the Meetup.

    5:45 – 6:15 PM    Rock with Spark - food and socializing with music by Scott Nichols

    6:15 – 6:45 PM    Big Data talk by Jonathon Gray, founder and CEO of Cask

    6:45 – 8:00 PM    Intro to Spark Structured Streaming

    Look forward to seeing you there.

    Spark Structured Streaming

    Structured Streaming is a new scalable and fault-tolerant stream processing engine built on the Spark SQL engine. Structured Streaming allows you to express your streaming computation the same way you would express a computation on static data. This has two benefits. The first is code reuse as essentially the same queries be run on batch, interactive or streaming data. Second, it simplifies streaming application development as you can operate on streams of data just like you can on static data using DataFrames. Structured Streaming abstracts away the complexity of streaming analytics allowing you to perform streaming analytics without having to reason about streaming.

    The Spark SQL engine takes care of running Structured Streaming queries and incrementally and continuously updating the result as streaming data continues to arrive. With Spark Structured Streaming, you can express streaming aggregations, event-time windows, as well as join streaming data to static data.

    In this session, we’ll walk through the basics of Structured Streaming, its programming model and APIs. The concepts will be illustrated using code examples. Then, we’ll walk through a demo of analyzing both static and streaming sensor data to show how the same queries can be used on each, thereby simplifying streaming analytics application development, and how static and streaming data can be leveraged together.


Join or login to comment.

  • Rich T.

    *** Meetup Cancelled due to Snow***

    Given the significant snow storm that we are getting tomorrow and the dangerous driving conditions it will create, I don't see any option but to reschedule our Meetup on Structured Streaming. We really appreciate all the interest and everyone who signed up to attend. It's pretty disappointing as we did a lot of work to pull this event together. We'll reschedule, dependent on the availability of the lecture hall.

    February 8

  • Michael F.

    Will the presentation be recorded and will the slides be made available? I don't know if I can get there in time. Thanks!

    February 5

    • Rich T.

      We are not planning to formally record the event. However, attendees have sometimes recorded sessions in the past. If someone wants to record the session and make it available, I'll post the link on this page.

      The slides and code/notebooks on Spark Structured Streaming will be made available after the session.

      February 6

  • Marina P.

    Could you let us know who is the speaker for the Spark Structured Streaming talk? Thanks!

    February 4

    • Rich T.

      I will be presenting the Spark Structured Streaming content.

      February 6

  • Jim M.

    will handouts be posted here? the location doesn't work. Thanks

    January 9

    • Rich T.

      Hi Jim. Yes. The slides and code/notebooks will be posted after the session.

      1 · January 12

Cancelled

Our Sponsors

  • Big Data University

    Free on-line courses on Hadoop and other big data technologies.

  • IBM

    Hands on experiences with IBM big data solutions

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy