September meetup

Hello everybody,

in september we will restart our meetups - after the summer break - with a super interesting presentation: a real life use case of Kafka, Airflow, Spark and Docker altogether used in building an internal flow of reporting. We have received many questions in the past regarding local use cases of Big Data and this time we will have a presentation that will contain an overview level ( from the issues to be solved to the steps taken forwards) but as well details about the components of the solution. Many thanks to Mihai and Ovidiu for taking time to build the presentation and come and present in our meetup.

Details of the agenda:

18:30 Getting together, socializing

19:00 - 19:40 Cloud pipeline for cleaning up internal reporting, Mihai Paraschiv (https://www.linkedin.com/in/mihaiparaschivro/) @ eMAG and Ovidiu Anicai (https://www.linkedin.com/in/ovimihai/) @eMAG

Accuracy and quality in reporting, for large e-Commerce companies like eMAG, can seriously be skewed by bot traffic. This affects every facet of decision making: forecasts, financial planning, commercial stocks, AB testing, etc. Identifying and filtering out bot traffic from reports requires a lot of signals, both internal and external. This presentation highlights how we integrated Kafka, Airflow, Spark and Docker in a hybrid setup (internal and Google Cloud) in order to create a complete pipeline (from raw data to results).

19:40 - 20:00 Q&A - I expect loads of questions for this presentation thus we will allow a longer Q&A time than usual

20:00 - 21:00 Socializing, pizza & drinks sponsored by our steady sponsor Netopia.

Look forward seeing you soon. Until then, have a great summer break,

Valentina