Building Goldman Sachs' next gen cluster management system

IMPORTANT: Venue changed, please check details

Talk abstract:

RPG Infra is a relatively small team at Goldman Sachs and we are looking after the firm’s job scheduling infrastructure and software distribution platform.

The scheduling infrastructure consists of several internally built products. One of them is called Procmon (“Process Monitor”) - a platform that processes millions of computing tasks per day. The 'computing tasks' executed by Procmon are typically scripts written in the proprietary language called Slang (which is itself is a part of the proprietary platform called SecDb) and usually involve some financial risk calculations.

About 12 months ago we started building a new task execution platform which is closely integrated with Procmon and allows us to schedule calculation tasks to a large pool of computing resources (we want to represent our data centres as a massive computer made up of thousands of cores and TBs of memory and grow/shrink it dynamically based on the load).

This talk will give a brief overview of this new platform and some of the goals and challenges that we are trying to solve by using Erlang and Riak_Core.

Speakers:
Roman Shestakov - VP at Goldman Sachs and a member of RPG Infra Team (RunTime Practices Team)
https://www.linkedin.com/in/roman-shestakov-6292065

Alex Tringham - Software Engineer, Enterprise Platforms at Goldman Sachs https://www.linkedin.com/in/alex-tringham-31a25963