What we're about

A while ago I entered the challenging world of Big Data. As an engineer, at first, I was not so impressed with this field. As time went by, I realised more and more, The technological challenges in this area are too great to master by one person. Just look at the picture in this articles, it only covers a small fraction of the technologies in the Big Data industry…

Consequently, I created a meetup detailing all the challenges of Big Data, especially in the world of cloud. I am using AWS & GCP and Data Center infrastructure to answer the basic questions of anyone starting their way in the big data world.

how to transform data (TXT, CSV, TSV, JSON) into Parquet, ORC,AVRO which technology should we use to model the data ? EMR? Athena? Redshift? Spectrum? Glue? Spark? SparkSQL? GCS? Big Query? Data flow? Data Lab? tensor flow? how to handle streaming? how to manage costs? Performance tips? Security tip? Cloud best practices tips?

In this meetup we shall present lecturers working on several cloud vendors, various big data platforms such hadoop, Data warehourses , startups working on big data products. basically - if it is related to big data - this is THE meetup.

Some of our online materials (mixed content from several cloud vendor):

Website:

https://big-data-demystified.ninja (under construction)

Meetups:

https://www.meetup.com/Big-Data-Demystified

https://www.meetup.com/AWS-Big-Data-Demystified/

You tube channels:

https://www.youtube.com/channel/UCMSdNB0fGmX5dXI7S7Y_LFA?view_as=subscriber

https://www.youtube.com/channel/UCzeGqhZIWU-hIDczWa8GtgQ?view_as=subscriber

Audience:

Data Engineers
Data Science
DevOps Engineers
Big Data Architects
Solution Architects
CTO
VP R&D

Upcoming events (5+)

Machine Learning Essentials | Big Data Demystified

Agenda 1800 gathering 1830 lecture 1: Fake News: between a new Front and a new Frontier, Nitzan -Or Kedrai[masked] Lecture 2: Machine Learning Essentials, Lior King Fake News: between a new Front and a new Frontier קוראים לי ניצן אור קדראי ואני עומדת בצומת המעניינת שבין טכנולוגיה, מדיה ואקטיביזם. בארבע וחצי השנים האחרונות אני עובדת בידיעות אחרונות, בהתחלה כמנהלת המוצר של אפליקציית ynet וכיום כמנהלת החדשנות. הייתי שותפה בהקמת עמותת סטארט-אח, עמותה המספקת שירותי פיתוח ומוצר עבור עמותות אחרות, ולאחרונה מתעסקת בהקמת קהילה שמטרתה לחקור את ההיבטים הטכנולוגיים של תופעת הפייק ניוז ובניית כלים אפליקטיביים לצורך ניהול חכם של המלחמה בתופעה. ההרצאה תדבר על תופעת הפייק ניוז. נתמקד בטכנולוגיה שמאפשרת את הפצת הפייק ניוז ונראה דוגמאות לשימוש בטכנולוגיה זו. נבחן את היקף התופעה ברשתות החברתיות ונלמד איך ענקיות הטכנולוגיה מנסות להילחם בה. Machine Learning Essentials Abstract: Machine Learning (ML) is one of the hottest topics in the IT world today. But what is it really all about? In this session we will talk about what ML actually is and in which cases it is useful. We will talk about a few common algorithms for creating ML models and demonstrate their use with Python. We will also take a peek at Deep Learning (DL) and Artificial Neural Networks and explain how they work (without too much math) and demonstrate DL model with Python. The target audience are developers, data engineers and DBAs that do not have prior experience with ML and want to know how it actually works.

Fake News: between a new Front and a new Frontier | Big Data Demystified

Agenda 1800 gathering 1830 lecture 1: Fake News: between a new Front and a new Frontier, Nitzan -Or Kedrai[masked] Lecture 2: Machine Learning Essentials, Lior King Fake News: between a new Front and a new Frontier קוראים לי ניצן אור קדראי ואני עומדת בצומת המעניינת שבין טכנולוגיה, מדיה ואקטיביזם. בארבע וחצי השנים האחרונות אני עובדת בידיעות אחרונות, בהתחלה כמנהלת המוצר של אפליקציית ynet וכיום כמנהלת החדשנות. הייתי שותפה בהקמת עמותת סטארט-אח, עמותה המספקת שירותי פיתוח ומוצר עבור עמותות אחרות, ולאחרונה מתעסקת בהקמת קהילה שמטרתה לחקור את ההיבטים הטכנולוגיים של תופעת הפייק ניוז ובניית כלים אפליקטיביים לצורך ניהול חכם של המלחמה בתופעה. ההרצאה תדבר על תופעת הפייק ניוז. נתמקד בטכנולוגיה שמאפשרת את הפצת הפייק ניוז ונראה דוגמאות לשימוש בטכנולוגיה זו. נבחן את היקף התופעה ברשתות החברתיות ונלמד איך ענקיות הטכנולוגיה מנסות להילחם בה. Machine Learning Essentials Abstract: Machine Learning (ML) is one of the hottest topics in the IT world today. But what is it really all about? In this session we will talk about what ML actually is and in which cases it is useful. We will talk about a few common algorithms for creating ML models and demonstrate their use with Python. We will also take a peek at Deep Learning (DL) and Artificial Neural Networks and explain how they work (without too much math) and demonstrate DL model with Python. The target audience are developers, data engineers and DBAs that do not have prior experience with ML and want to know how it actually works.

80% Cost Reduction in Google Cloud BigQuery - WTF? | Tips and Tricks

Agenda: 1800 - gathering 1830 - 80% Cost reduction in BigQuery - WTF? | Tips and Tricks, Omid Vahdaty, Big Data Ninja 1900 - 2030 -TBD, stay tuned :) date is not final yet. be advised, this event is right after the holidays. What will you learn? "Cost reduction in BigQuery - WTF? | Tips and Tricks" In this lecture I will share with how I saved 80% of BigQuery monthly billing of investing.com

Airflow Demystified | Big Data Demystified

investing

1800 Gathering and Networking 1830 - 1900 "Airflow demystified" , Omid Vahdaty Big Data Ninja 1900 - 2100 "Couchbase data platform", Lior King, Couchbase Event Date is not final What will you learn? 1. "Airflow demystified" in this lecture i will share with you the basics of creating ETLs via Airflow 2. "Couchbase data platform" Couchbase is a popular open source NoSQL platform used by giants like Apple, LinkedIn, Walmart, Visa and many others and runs on-premise or in a public/hybrid/multi cloud. Couchbase has a sub-millisecond K/V cache integrated with a document based DB, a unique and many more services and features. In this session we will talk about the unique architecture of Couchbase, its unique N1QL language - a SQL-Like language that is ANSI compliant, the services and features Couchbase offers and demonstrate some of them live. We will also discuss what makes Couchbase different than other popular NoSQL platforms like MongoDB, Cassandra, Redis, DynamoDB etc. At the end we will talk about the next version of Couchbase (6.5) that will be released later this year and about Couchbase 7.0 that will be released next year. Time: 120 minutes (70 minutes, a break and 50 minutes).

Photos (30)