June Meeting – Analyzing Rat Brain Neuronal Signals with Hadoop and Hive

If you're looking for applications of big data technology beyond ad optimization and e-commerce, then be sure to join us for the June meeting of Chicago Big Data. We'll hear from Brad Rubin, Associate Professor at the University of St. Thomas, on the benefits and tradeoffs of leveraging Hadoop in a scientific application. More details are below. Note that the location is TBD, but we'll update as soon as that's finalized. Look forward to seeing everyone!

Abstract

Although Hadoop and other Big Data technologies are typically applied to I/O intensive workloads, where parallel data channels dramatically increase I/O throughput, there is growing interest in applying these technologies to CPU intensive workloads. In this work, we used Hadoop and Hive to digitally signal process individual neuron voltage signals captured from electrodes embedded in the rat brain. Previously, this processing was performed on a single Matlab workstation. With Hadoop/Hive, we were not only able to apply parallelism to the various processing steps, but had the benefit of having all the data online for additional ad hoc analysis. We describe the tradeoffs between the Matlab and Hadoop/Hive approach, performance results, and several issues identified with using Hadoop/Hive in this type of application.

Bio

Brad Rubin is an Associate Professor at the University of St. Thomas in St. Paul, MN in the Graduate Programs in Software department where he teaches courses in Big Data, Information Retrieval, Computer Security, and Software Analysis and Design. He is the co-founder of the Center of Excellence for Big Data. Previously, he spent 14 years with IBM in Rochester, MN, working on all facets of the AS/400 hardware and software development, starting with its first release. He was a key player in IBM's move to embrace the Java platform, and was lead architect of IBM's largest Java application, a business application framework product called San Francisco (now part of WebSphere). He was also chief technology officer for the Data Storage and Information Management division of Imation Corp., as well as the leader of its R&D organization. Dr. Rubin has degrees in Computer and Electrical Engineering from the University of Illinois, Champaign-Urbana, and a Doctorate in Computer Science from the University of Wisconsin, Madison. He holds five patents and four invention disclosures, has authored many research and trade publications, consults, and is a frequent speaker on technology topics.