9th Spark London Meetup : Using Numerical Libraries on Spark

For this August meetup Brian Spector from the Numerical Algorithms Group (NAG) will be discussing using numerical libraries on Spark.

Brian is a Technical Consultant at NAG where he has begun to successfully implement the NAG Library’s 1600 mathematical routines for Big Data applications. Brian will share the many pitfalls and successes he has had while using a numerical library in a distributed computing environment.

The talk will start at 7. From 6:30 usual networking and goods.

This meetup is sponsored by SoftLayer.

http://photos2.meetupstatic.com/photos/event/1/8/2/a/600_440706186.jpeg

Title: Using Numerical Libraries on Spark

Synopsis:

For efficient numerical computations, today’s algorithms require all relevant data to be in-memory at run time. As this differs from the Hadoop ecosystem, we must now rethink our existing algorithms and reformulate problems for large scale applications. During this talk we will review both past and current techniques for solving a multi-linear regression problem. We will show how the past algorithms break down and how to efficiently reformulate this problem into Apache Spark’s MapReduce context by solving the Normal Equations. We’ll touch on the efficient algorithms for Big Data applications and the importance of scaling as you increase the number of worker nodes. Other topics covered include environment setup, debugging worker nodes, and partition sizes on Spark.

SoftLayer

SoftLayer, an IBM Company, operates a global cloud infrastructure platform built for Internet scale. With more than 180,000 devices under management, and a global footprint of data centers and network points of presence, SoftLayerprovides Infrastructure-as-a-Service to leading-edge customers ranging from Web startups to global enterprises. SoftLayer’s modular architecture provides unparalleled performance and control, with a full-featured API and sophisticated automation controlling a flexible unified platform that seamlessly spans physical and virtual devices, and a worldwide network for secure, low-latency communications. For more information, please visit softlayer.com (http://softlayer.com/).