Massive Log and Event Data Management and Monitoring with Apache Flume

  • May 18, 2013 · 12:00 PM
  • This location is shown only to members

Israel Ekpo will guide a presentation on

Massive Log and Event Data Management and Monitoring with Apache Flume
  • How to stream in data from sources like log files, Twitter, Google+, Facebook, Avro, Syslog, Netcat, SQL-powered relational databases.
  • How to configure the sources, interceptors, channels and sinks in Flume agents to pick up events and send them to centralized data stores such ElasticSearch, Apache Solr, HDFS, HBase, MongoDB etc.
  • How to search through the index and find things easily using features available in ElasticSearch and Apache Solr.
  • How to maintain the indexed data over time as it grows.
  • Best practices for configuration, logging, search performance and index maintainance.

 

Join or login to comment.

  • Scott

    Very informative and fun

    May 19, 2013

  • Israel E.

    Thank you Lillian and Wayne for putting this together.

    Great effort.

    Thank you everyone for coming to the talk.

    It is my hope that this introduction will inspire you to look into how Apache Flume, ElasticSearch or D3.js can benefit current and future projects you embark on.

    I keep it simple because I did not want to go into two much details.

    The slides are available here:

    https://speakerdeck.com/bigdata1024/massive-data-aggregation-processing-monitoring-and-visualization-with-apache-flume-elasticsearch-and-d3-dot-js

    If you have any questions or comments, all my contact information are on slide number 2.

    Thanks again.

    1 · May 19, 2013

    • A former member
      A former member

      Thanks for positing the slides. I missed the talk because of BarCamp but after reading your slides and doing a little research I was able to determine if Flume was something we should consider using (probably not).

      May 19, 2013

  • Israel E.

    It was a great experience. I had an opportunity to meet with very smart entrepreneurs and like minded people.

    May 19, 2013

  • Lillian P.

    Israel is incredible... He is a great speak and explains very technical things so that they seem simple. He is generous with his time and with sharing his expertise. We were lucky to have him speak :)

    May 18, 2013

  • Andrew B.

    Great introduction to Flume and Elasticsearch -- thanks to Israel for putting together the talk!

    May 18, 2013

  • Pablo M.

    2 year old nephew in town from Boston, will not be able to attend.

    May 18, 2013

  • Kym G.

    Daughter graduates that weekend from HS.

    April 14, 2013

    • Lillian P.

      Thats wonderful - enjoy that experience. We will miss you!

      May 18, 2013

  • A former member
    A former member

    Really would love to start coming to these meetups. However, they seem to keep getting scheduled on days where other large events are happening.

    May 18th, for example is BarCamp Orlando (http://barcamporlando.org/). I believe the last meetup was during Startup Weekend...

    Is there any way for future meetups that other events can be taken in to consideration? For reference, this newsletter is sent out monthly consisting of all the tech events for that month: http://techevents.us/orlando/

    Thanks for your consideration :)

    April 25, 2013

    • Wayne S.

      I'd love your input/help for topics/dates/locations for future meetups. Please add them to the discussion thread on future meetups.

      April 25, 2013

    • Lillian P.

      Jesse, thanks for this resource - we are going to try some alternative meetup means to be able to accommodate around these other important events. We will miss you!

      May 18, 2013

  • A former member
    A former member

    I agree with Jesse: I'm not going to attend because BarCamp is the same day.

    1 · April 25, 2013

    • Lillian P.

      We will try to schedule better in the future

      May 18, 2013

  • Israel E.

    I know the purpose of meetups like this is to meet people in person and network. Has anyone though about meeting virtually using GoToWebinar, WebEx, Join.me or Google Hangout?

    Being able to meet virtually could increase participation and eliminate the need to travel, parking fees etc.

    I will provide folks that are not able to meet to get together at other times later in the evenings or afternoon.

    Any thoughts?

    1 · April 25, 2013

    • Lillian P.

      This is a good idea - We do this with Ushahidi bc it is an international Meetup and it works fine... they meetup via Skype... lets talk about it at the meeting

      May 18, 2013

  • Dustin

    Oops. I thought this was for Sunday; Ill be in Canada on the 18th. Wish I could be there.

    May 12, 2013

  • Israel E.

    Regarding this particular event on May 18th can we put up something on Survey Monkey and poll members to know what other days will work?

    We could put a couple of weekend dates up for voting on Survey Monkey to get some feedback.

    April 25, 2013

    • Wayne S.

      the challenge is ensuring a venue. Can someone can offer a dedicated venue with date flexibility for this or future meetups?

      April 25, 2013

15 went

Our Sponsors

People in this
Meetup are also in:

Create a Meetup Group and meet new people

Get started Learn more
Allison

Meetup has allowed me to meet people I wouldn't have met naturally - they're totally different than me.

Allison, started Women's Adventure Travel

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy