BookKeeper Nov 2016 Meetup

Agenda

• 6:00 - 6-30 - Socialize over food and beers

• 6:30 - 7:00 - Reliable Data Replication using Apache DistributedLog/BookKeeper

• 7:00 - 7:30 - Sherpa - Distributed NoSQL database

• 7:30 - 8:00 - Apache BookKeeper as a longterm distributed store

Session 1 : Reliable Data Replication using Apache DistributedLog/BookKeeper

Apache DistributedLog (incubating) is a low-latency, high throughput replicated log service. It is built on top of Apache BookKeeper, providing end-less log-streams as the real-time storage primitive. It offers durability, replication and strong consistency as essentials for building reliable real-time data services. It has been used widely at Twitter in applications from transactional key-value store to real-time pub/sub messaging and reliable data replication.

Sijie Guo will give an overview of Apache DistributedLog, detailing how DistributedLog can be used for building reliable data replication into any data services. This talk will discuss two major use cases at Twitter: how DistributedLog is used by Manhattan (Twitter's distributed key/value store) to support strong consistency and achieve reliable cross-datacenter replication and how DistributedLog is used for disaster recovery in pub/sub messaging.

Sijie Guo is a staff software engineer at Twitter, where he is the tech lead of Messaging team. He is one of the authors of Apache DistributedLog (incubating) and the PMC chair of Apache BookKeeper.

Session 2 : Sherpa - Distributed NoSQL database

Sherpa is a distributed NoSQL databased that has arisen from the Yahoo Labs PNUTS paper. It provides a hosted multi-tenant database with REST interface and it operates on semi-structured data. A particular characteristic of Sherpa is its use of a pub/sub message bus as the transport for data and control messages, guaranteeing data durability and replication within the same region as well as across different geographic regions.

The messaging layer has been recently migrated to use Pulsar, a pub/sub system recently open-sourced by Yahoo, that relies on Apache BookKeeper for the data storage and replication.

Maurice Barnum is an architect at Yahoo and he is the tech lead for Sherpa.

Session 3: Apache BookKeeper as a longterm distributed store

Salesforce is building low-latency high-throughput distributed long-term storage on Apache BookKeeper. This store is used by highly interactive and data intensive salesforce applications. These apps need quick response from back-end store. Where a single request may result into multiple storage round trips.

Salesforce is enhancing Apache BookKeeper for this workload and actively participating and contributing back to the community.During this talk we will go over lessons learned through our journey, along with current and proposed future enhancements.

Venkateswararao Jujjuri (JV), Cloud Storage Architect at Salesforce who is responsible for building an elastic, highly available, highly scalable multi petabyte distributed storage services on commodity hardware. Actively working with multiple opensoruce solutions including Apache BookKeeper.

Andrey Yegorov is a PMTS at Salesforce where he works on multi petabytes distributed storage systems. In the past he built near-realtime distributed processing system and also worked on web search.