Singapore Open Source Data Infrastructure Meetup - November 2023

✅ Are you interested in learning more about open-source data technologies?
✅ Do you want to network with other like-minded people in a fun, relaxed environment?

Then come join us on Thursday, 16 November for an evening filled with great opportunities for networking, and talks. Read below for more details!

Refreshments* will be provided before the talks, so be sure to come early to have a bite and network with your fellow peers.

Note: The event starts at 5:30PM SGT and runs until 8:30PM SGT.

Program:
5:30PM - Open Doors + Registration (and refreshments)
6:00 PM - Welcome
6:10 PM - Beginners guide to balance your data across Apache Kafka partitions | Olena Kutsenko, Senior Developer Advocate at Aiven

• Abstract: Apache Kafka is a distributed system. At the heart of Apache Kafka is a set of brokers that contain topics. Topics are split into partitions. Dividing topics into smaller pieces allows us to work with data in parallel and achieve higher data throughput.

Such parallelization is the key to a performant cluster, however it comes with a price. First, reading from multiple partitions will eventually mess up the order of records, meaning that the resulting order will be different from when the data was pushed into the cluster. Another big challenge is uneven distribution of data across partitions.

Overloaded partitions present a dangerous issue for performance of all involved parties, but especially for brokers and consumers. Therefore, when building our product architecture we should carefully weigh up how many partitions we need, how to ensure proper message ordering, how to balance records across partitions, not forgetting about data load distribution over time. And do all of this while still maintaining good performance of the cluster.

If you're fresh to Apache Kafka, or looking for good practices to design your partitions and avoid common pitfalls, you'll find this session useful!

• Speaker: Olena is a seasoned expert in data, sustainable software development, and teamwork. With a background in software engineering, she's led teams and developed mission-critical applications at Nokia, HERE Technologies, and AWS. Currently, she works at Aiven where she supports developers and customers in using open-source data technologies such as Apache Kafka, ClickHouse, and OpenSearch. She is also an international public speaker and regularly presents at conferences around the world. She holds AWS Developer and Solutions Architect certifications, and is also a Confluent Catalyst.

6:40 PM - Design Considerations for Hosting Data Infrastructure | Shek Chian Low, Global Solution Architect at UpCloud

• Abstract: SC talks about infrastructure design considerations when building a system for hosting databases and data infra in general, covering both self managed and managed offerings.
• Speaker: Shek Chian (SC) is UpCloud’s Global Solution Architect. Boasting experience from principal product companies and consultancies in Singapore, he consults and assists end-users across Europe and Asia Pacific to find creative solutions to optimise their infrastructure.

7:10 - 8:30 PM - Socialising

*Please note that this is an alcohol-free event.