October SF Hadoop Users Meetup

The September SF Hadoop User Group meetup will be held Wednesday, October 8 from 6:00pm to 8:00pm. This meetup will be hosted by MapR and located at WibiData, 375 Alabama Street #350, San Francisco. Food and drinks will be served.

Presentation: "Self Service Data Exploration with Apache Drill"

Tomer Shiran - VP, Product Management, MapR

Time to value is everything.

With the emergence of new data sources such as web logs, social media, mobile applications and sensor data, organizations are looking extend BI by providing insights into new areas such as operational performance, product optimization and customer satisfaction. However, traditional data management processes simply don’t work in this new world of big data. Organizations now not only have to manage higher volumes of information, but the data itself arrives at faster rates in real time and is more complex and dynamic than traditional transactional datasets. To be useful, this data must be analyzed in much shorter intervals than traditional reporting cycles of weeks and months.

In this session, we will see how Apache Drill is driving this audacious goal to bring Self Service data exploration natively on the Hadoop/NoSQL data and how it lets users explore any type of data immediately as it comes in using the familiar ANSI SQL language/SQL tools they are already familiar with. It also covers the exciting challenges that the Apache Drill community is working with, its current status and the roadmap.

Presentation: "How to Find What You Didn't Know to Look For, Practical Anomaly Detection"

Ted Dunning - Chief Application Architect, MapR

Anomaly detection is the art of automating surprise. To do this, we have to be able to define what we mean by "normal" and recognize what it means to be different from that.

The basic ideas of anomaly detection are simple. You build a model and you look for data points that don’t match that model. The mathematical underpinnings of this can be quite daunting, but modern approaches provide ways to solve the problem in many common situations.

I will describe these modern approaches with particular emphasis on several real use-cases including:

• a) rate shifts to determine when events such as web traffic, purchases or process progress beacons shift rate

• b) time series generated by machines or biomedical measurements

• c) topic spotting to determine when new topics appear in a content stream such as Twitter

• d) network flow anomalies to determine when systems with defined inputs and outputs act strangely.

In building a practical anomaly detection system, you have to deal with practical details starting with algorithm selection, data flow architecture, anomaly alerting, user interfaces and visualizations. I will show how to deal with each of these aspects of the problem with an emphasis on realistic system design.