Sydney Data Eng meetup, September 2024

Hey all, please join us for our September Sydney Data Eng meetup. Join us for an engaging session filled with exciting discussions and networking opportunities.

Don't miss out—RSVP to secure your spot. A big shout-out to Rokt for providing us with the venue and catering! 🎉

🏠 Location: Rokt office, Level 16/300 Elizabeth St, Surry Hills NSW 2010, Australia
🍕 Catering: Rokt 😊
💬 Join our Slack Group here: Data Engineers Slack Channel

Schedule:
- 5.30pm: Doors Open
- 6.00pm: Announcements and Welcome
- 6.10pm: 1st Talk
- 6.30pm: 2nd Talk
- 6.50pm: 3rd Talk
- 7.10pm: Networking
- 8.00pm: Doors Close

Speakers:
🎤 Johnny Mirza, Solution Architect at ClickHouse
Talk Title: What’s new with ClickHouse: Updates, Integrations & Acquisitions
Talk Summary: With tons of feature updates and performance optimisations in 24.x Keeping up with the rapid pace of feature updates and performance optimisations in ClickHouse 24.x can be challenging. In this talk, we’ll spotlight the top 15 enhancements, integrations, and performance improvements that you need to know about. We’ll also introduce you to the new tools developed by our engineering and training teams designed to boost your success with ClickHouse. Additionally, we’ll delve into our recent strategic acquisitions—chDB and PeerDB—exploring the reasons behind these moves and how they fit into the ClickHouse ecosystem. Join us to stay informed and make the most of ClickHouse’s latest advancements!

Speaker Bio: Johnny Mirza is a Solution Architect with ClickHouse, working with users across APAC. With over 20 years of background in solutions engineering, he’s experienced in architecting and enabling solutions for enterprise clients in the telecommunications, media, insurance, and financial services sectors. Johnny has a high level of expertise of integration between both public cloud and on-premise infrastructure, while focussing on service assurance, monitoring platforms, and open-source technologies. Prior to ClickHouse, Johnny was part of the solution engineering teams at Confluent, Splunk, and Optus, to name a few.

🎤 Sharat Madanapalli, Head of Data & AI at Canopus
Talk Title: Our Journey to Ingest and Analyse 5 Billion Network Events / Day
Talk Summary: In this talk, Sharat will share his team’s journey in handling massive-scale network event data, processing 5 billion events daily using ClickHouse. Starting with CSV-based reporting, they transitioned through TimescaleDB and ultimately adopted ClickHouse. The talk will cover scaling strategies, including Kafka table implementation, optimising primary keys, materialised views, and experiments with JSON, map data types, and distributed ClickHouse.
Sharat will also compare their Kafka ingestion with a custom Go-based solution and discuss their use of DBT and Prefect for data rollups. Key takeaways will include system table usage, monitoring, and optimising slow queries, along with lessons from the ClickHouse community.

Speaker Bio: As the head of data engineering and AI at Canopus, with a PhD and six years of industry experience spanning software, data, AI, Sharat will share his team’s path from starting with simple CSV dumps to a sophisticated, high-performance analytics platform.

🎤 Dan Draper, Founder & CEO at CipherStash
Talk Title: Searchable encryption in-use for ClickHouse: what, how and why
Talk Summary: Encryption is a powerful way to keep data protected…but maybe not in the way you think! You may be familiar with traditional at-rest encryption but in this talk, Dan will introduce the concept of encryption in use and how to use it for granular access control in ClickHouse.
The talk will explain how encryption in use can be made practical for large datasets while maintaining high levels of performance, as well as how full query functionality can be maintained through the use of searchable encryption.
The talk should leave you with a new perspective on encryption and the role it can play in keeping data safe with affecting usability. Dan will share some tips and resources for further exploration.
Speaker Bio: Dan is the CEO and founder of CipherStash which provides a highly secure, frictionless data protection platform that works everywhere sensitive data is stored. Dan is a life-long techie, nerd and is proudly neuro-spicy having been diagnosed with ADHD at the age of 40. Dan’s obsession for the past 8 years has been cryptography and the powerful ways it can be used for data protection and privacy.

If you'd like to present at future meetups, please submit your talk here!

Remember to bring along some great questions! See you all there...