Kafka at the BBC & Kafka ops at Sky B&G

Join us for the 2nd Leeds Apache Kafka meetup on May 29th from 6:00pm hosted by Sky Betting and Gaming in Leeds. The agenda, venue and speaker information can be found below. See you there!

-----

Agenda:
6:30pm: Doors open
6:30pm - 7:00pm: Pizza & Drinks & Chat
7:00pm - 7:45pm: Please can I have a Kafka Cluster? Meeting the operational scale of Kafka - David Sunter, Sky Betting and Gaming
7:45pm - 8:30pm: As Above, So Below – Building Microservice Pipelines with Kafka and Clojure - Dan Meyers, BBC
8:30 pm - 9:00pm: Additional Q&A and Networking

-----

• Title: Please can I have a Kafka Cluster? Meeting the operational scale of Kafka
• Speaker: David Sunter, Sky Betting and Gaming.
• Abstract: How Platform Services have gone from supporting 4 Kafka Clusters in 2017, to creating 9 Clusters so far in 2018. Scaling the amount of Clusters to met the demand of SBG and how it's become an integral part of the company's future.
• Bio: David joined SBG in 2016, after background in aerospace engineering. After exploring a range of roles in devops, across SBG, David joined Platform Services where for the last year he has supported Kafka as a service. Kafka is now a key component in the running of real time services for both the Betting and Gaming tribes. David has interests in statistics, analytics and asking the question why? Outside of work he is a passionate sports fan and dog lover.

-----

• Title: As Above, So Below – Building Microservice Pipelines with Kafka and Clojure
• Speaker: Dan Meyers, BBC
• Abstract: In the beginning was PIPs, an API backed by a relational database used to store all the BBC's programme metadata. But as more clients came, with more requirements and ever more complex queries, the system became unwieldy to develop and use. The Programme Metadata team turned to a combination of a Kafka platform and Clojure microservices running in AWS to produce multiple pipelines, one per client requirement. This setup turns the normal ETL pipeline on its head, with one consistent backend and multiple differing outputs. This talk will show some common patterns used by the team, how they use the microservice approach to quickly spin up new client pipelines, and how the structure of Clojure mirrors and supports the way they use Kafka.
• Bio: Dan graduated in 2004 and spent 10 years working as a Network Engineer. In 2014, realising that the bits of the job he enjoyed the most all involved coding, he changed tack to become a Software Engineer for the BBC in Manchester, UK. He is now a Senior Software Engineer in the Programme Metadata team, delivering scalable and reliable APIs used as part of the backend to the BBC's programme distribution through iPlayer, /programmes, schedule EPGs, and third parties like Sky and Virgin's on-demand services or the American "BritBox" streaming platform. He has the world's best Labrador.

-----

The full address for the venue is:

Sky Betting & Gaming
3rd Floor
6 Wellington Place
Leeds
LS1 4AP

For people who have visited the Sky Betting & Gaming offices previously, this is our new building.

There is wheelchair access. There is basic seating, an accessible toilet, a gender neutral toilet, and general car parking. There is a designated quiet space by request in advance. This isn’t a hearing loop, or a bsl interpreter. There is a level route from the NCP car park (which has limited accessible parking spaces), CitiPark, and the train station. The venue is on the 3rd floor with multiple lifts available.

Car parking: http://www.wellingtonplace.co.uk