Effective Data Integration Patterns

In October we're hosting a special Bucharest Big Data meetup, together with our friends from Cloud Native Bucharest, as part of Big Data Week.
**Please note you don’t need a BDW conference ticket to attend this event, however, you need to RSVP here to secure your spot.**

Speakers
Our awesome speakers for the evening are:
Andrei Fecioru, Senior Software Engineer & Eduard Ciurezu, Software Engineer @ Adobe will look at a series of techniques and practices of working with a high-throughput batch ETL pipeline using the Spark distributed computing framework, addressing topics such as:
– How to deal with input data structures;
– How to shape data for improved performance;
– Tradeoffs in data access patterns;
– Stability of large computing clusters;
– How to reduce computing footprint in order to achieve significant operating cost reductions.

Razvan Mantu, Senior Data Engineer & Alex Giurgiu, Data Engineering Manager @ Adore Me will delve into their innovative approach to integrating microservice data into a BigQuery Data Warehouse (DWH). They’ll uncover how they streamline the process, allowing backend engineers to publish new data seamlessly through JSON schema files and pull requests, ensuring precise and automated data flow, covering the following:
- Seamless integration of microservice data into BigQuery DWH;
- Utilization of Pub/Sub and Cloud Functions for efficient data ingestion;
- Advanced data manipulation using stored procedures and user-defined functions;
- Automatic code and documentation generation from JSON schema;
- Streamlined CI/CD for automatic provisioning from GitHub.

Agenda
18:30 - 19:00 - Welcome
19:00 - 19:40 - Andrei Fecioru & Eduard Ciurezu: Optimizing large-scale batch ETL pipelines in Adobe Experience Platform
19:40 - 20:20 - Razvan Mantu & Alex Giurgiu: Efficient Data Ingestion and SQL Generation with BigQuery and JSON Schema
20:20 - 21:30 - Networking

Meet us all on Thursday, October 5, at eSolutions Academy for engaging discussions on advanced data strategies.
**Presentations will be conducted in Romanian.**

See you there!
—
Big Data Week Bucharest Conference lines up top industry speakers presenting a wide array of topics from big data architectures, cloud, and infrastructure, to BI, analytics, AI, and ML. Check it out: https://bucharest.bigdataweek.com/
As a partner community, we get 25% off conference passes for the anchor conference on October 3-4, at Sheraton Hotel, using the code BDWMTP25.