What we're about

A while ago I entered the challenging world of Big Data. As an engineer, at first, I was not so impressed with this field. As time went by, I realised more and more, The technological challenges in this area are too great to master by one person. Just look at the picture in this articles, it only covers a small fraction of the technologies in the Big Data industry…

Consequently, I created a meetup detailing all the challenges of Big Data, especially in the world of cloud. I am using AWS & GCP and Data Center infrastructure to answer the basic questions of anyone starting their way in the big data world.

how to transform data (TXT, CSV, TSV, JSON) into Parquet, ORC,AVRO which technology should we use to model the data ? EMR? Athena? Redshift? Spectrum? Glue? Spark? SparkSQL? GCS? Big Query? Data flow? Data Lab? tensor flow? how to handle streaming? how to manage costs? Performance tips? Security tip? Cloud best practices tips?

In this meetup we shall present lecturers working on several cloud vendors, various big data platforms such hadoop, Data warehourses , startups working on big data products. basically - if it is related to big data - this is THE meetup.

Some of our online materials (mixed content from several cloud vendor):

Website:

https://big-data-demystified.ninja (under construction)

Meetups:

https://www.meetup.com/Big-Data-Demystified

https://www.meetup.com/AWS-Big-Data-Demystified/

You tube channels:

https://www.youtube.com/channel/UCMSdNB0fGmX5dXI7S7Y_LFA?view_as=subscriber

https://www.youtube.com/channel/UCzeGqhZIWU-hIDczWa8GtgQ?view_as=subscriber

Audience:

Data Engineers
Data Science
DevOps Engineers
Big Data Architects
Solution Architects
CTO
VP R&D

Upcoming events (5)

AWS Big Data Demystified | Big Data Architecture Lesson learned 1.2

This meetup is a cross meetup session, we are hosting Guy Glantser and his meetup, and I will be a guest lecture. I just inviting my meetup as well. Agenda: • 18:00-18:30 - Gathering, Networking, Hugs and Kisses • 18:30-18:45 - Opening, Announcements, and More... • 18:45-20:15 - First Session (AWS Big Data Demystified) • 20:15-20:30 - Break (More Networking) • 20:30-21:00 - Second Session (Azure Data Services – The Latest Updates & Announcements) Sessions: 1. AWS Big Data Demystified – Omid Vahdaty (90 Minutes) A while ago I entered the challenging world of Big Data. As an engineer, at first, I was not so impressed with this field. As time went by, I realized more and more. The technological challenges in this area are too great to master by one person. Consequently, I created a meetup detailing all the challenges of Big Data, especially in the world of cloud. I am using AWS infrastructure to answer the basic questions of anyone starting their way in the big data world: • How to transform data (TXT, CSV, TSV, JSON) into Parquet, ORC… • Which technology should we use to model the data? EMR? Athena? Redshift? Spectrum? Glue? Spark? SparkSQL? • How to handle streaming? • How to manage costs? • Performance tips? • Security tips? • Cloud best practices tips? 2. Azure Data Services – The Latest Updates & Announcements – Guy Glantser (30 Minutes) In this session we will cover the latest updates, releases and announcements in Azure related to data services and platforms, such as: Azure SQL Database, Azure Databricks and Power BI.

Big Data Demystified | Introduction to Azure Machine Learning

Agenda: 18:00 gathering and networking 18:30 "Introduction to Azure Machine Learning" , Guy Glantser, CEO Madeira Data Solutions 19:15 "From Redshift to SnowFlake", Yaron Tomer is VP R&D of XMPie, "Introduction to Azure Machine Learning" abstract: Machine Learning is currently one of the hottest buzzwords in the high-tech industry. It’s the science of getting computers to act without being explicitly programmed. The idea behind it is to train the system by learning from historical data, and produce a program that can predict future behavior. You probably use Machine Learning dozens of times a day without even knowing about it, like when you search the web, buy something in Amazon, or even when you go through your feed in Facebook. Azure offers a fully managed Machine Learning cloud service that enables you to easily build, deploy, and share predictive analytics solutions. In this session we will learn what Machine Learning is and why we should use it, how it can be used to analyze historical data and predict future behavior, what some of the business use cases for Machine Learning are, and how it all works in Azure. The session includes some cool examples that will demonstrate the power of Machine Learning. Guy Glantser, bio: Guy Glantser, Data Platform MVP, is the leader of the Israeli PASS local group and also the CEO and founder of Madeira Data Solutions. His career has been focused on the Microsoft Data Platform for the past 20 years, performing various database roles as either an on-site DBA, an external consultant or a speaker. Guy is involved in many activities in the Microsoft Data Platform community. He occasionally speaks at community events, such as PASS Summit, SQLBits, SQL Saturdays and user groups around the world. He also co-hosts the SQL Server Radio podcast. "From Redshift to SnowFlake", Yaron Tomer is VP R&D of XMPie: In the presentation I will briefly take you through our journey of selecting of the best data warehouse for our product, which ended up being Snowflake. We will then deep dive into Snowflake. You will learn about the amazing features and architecture of Snowflake that overwhelmed us and that caused us to understand it is the best solution for us on so many levels. It is rare to see a product so polished. I think the word must be spread, because although Snowflake is quite popular in the valley, only a dozen companies adopted it in Israel. About the lecturer: Yaron Tomer is VP R&D of XMPie, providing a cloud based marketing automation product to marketing agencies and fortune 500 enterprises. Before becoming a VP, Yaron managed the development of this SaaS product, from an idea to a mature product. Yaron is passionate or products and technologies.

Big Data Demystified | From Redshift to SnowFlake

Investing.com

Agenda: 18:00 gathering and networking 18:30 "Introduction to Azure Machine Learning" , Guy Glantser, CEO Madeira Data Solutions 19:15 "From Redshift to SnowFlake", Yaron Tomer is VP R&D of XMPie, "Introduction to Azure Machine Learning" abstract: Machine Learning is currently one of the hottest buzzwords in the high-tech industry. It’s the science of getting computers to act without being explicitly programmed. The idea behind it is to train the system by learning from historical data, and produce a program that can predict future behavior. You probably use Machine Learning dozens of times a day without even knowing about it, like when you search the web, buy something in Amazon, or even when you go through your feed in Facebook. Azure offers a fully managed Machine Learning cloud service that enables you to easily build, deploy, and share predictive analytics solutions. In this session we will learn what Machine Learning is and why we should use it, how it can be used to analyze historical data and predict future behavior, what some of the business use cases for Machine Learning are, and how it all works in Azure. The session includes some cool examples that will demonstrate the power of Machine Learning. Guy Glantser, bio: Guy Glantser, Data Platform MVP, is the leader of the Israeli PASS local group and also the CEO and founder of Madeira Data Solutions. His career has been focused on the Microsoft Data Platform for the past 20 years, performing various database roles as either an on-site DBA, an external consultant or a speaker. Guy is involved in many activities in the Microsoft Data Platform community. He occasionally speaks at community events, such as PASS Summit, SQLBits, SQL Saturdays and user groups around the world. He also co-hosts the SQL Server Radio podcast. "From Redshift to SnowFlake", Yaron Tomer is VP R&D of XMPie: In the presentation I will briefly take you through our journey of selecting of the best data warehouse for our product, which ended up being Snowflake. We will then deep dive into Snowflake. You will learn about the amazing features and architecture of Snowflake that overwhelmed us and that caused us to understand it is the best solution for us on so many levels. It is rare to see a product so polished. I think the word must be spread, because although Snowflake is quite popular in the valley, only a dozen companies adopted it in Israel. About the lecturer: Yaron Tomer is VP R&D of XMPie, providing a cloud based marketing automation product to marketing agencies and fortune 500 enterprises. Before becoming a VP, Yaron managed the development of this SaaS product, from an idea to a mature product. Yaron is passionate or products and technologies.

AI & Big Data in Health Sector-Opportunities & challenges | Big Data Demystified

18:00 networking 18:30 "AI & BIG DATA IN HEALTH SECTOR-OPPORTUNITIES & SECURITY / PRIVACY CHALLENGES", Alexander Raif, Chief security and Privacy architect at Maccabi Health care HMO. [Spoken language - English ] 19:15 break 19:20 "NoSQL at Extreme Performance - taking your NoSQL to the Next Level" Zohar Elkayam, Solutions Architect at Aerospike ETA to FINISH 21:00 "AI & BIG DATA IN HEALTH SECTOR-OPPORTUNITIES & SECURITY / PRIVACY CHALLENGES" Lecturer has Deep experience defining Cloud computing, security models for IaaS, PaaS, and SaaS architectures specifically as the architecture relates to IAM. Deep Experience Defining Privacy protection Policy, a big fan of GDPR interpretation. DeelExperience in Information security, Defining Healthcare security best practices including AI and Big Data, IT Security and ICS security and privacy controls in the industrial environments. Deep knowledge of security frameworks such as Cloud Security Alliance (CSA), International Organization for Standardization (ISO), National Institute of Standards and Technology (NIST), IBM ITCS104 etc. What Will You learn: Every day, the website collects a huge amount of data. The data allows to analyze the behavior of Internet users, their interests, their purchasing behavior and the conversion rates. In order to increase business, big data offers the tools to analyze and process data in order to reveal competitive advantages from the data. What Healthcare has to do with Big Data How AI can assist in patient care? Why some are afraid? Are there any dangers? "NoSQL at Extreme Performance - taking your NoSQL to the Next Level" Zohar Elkayam, Solutions Architect at Aerospike Bio: Zohar is a technology evangelist and an expert in data technologies with over 20 years of experience. At his current position, Zohar is a Solutions Architect for Aerospike - a low latency-high throughput big scale database. Before that, Zohar was an Oracle ACE, CTO and consultant technical lead at Brillix, and the director for databases, big data and BI at Glasshouse Technologies. Abstract/description: Building a low latency (sub millisecond), high throughput database that can handle big data AND linearly scale is not easy - but we did it anyway... In this session we will get to know Aerospike, an enterprise distributed primary key database solution. - We will do an introduction to Aerospike - basic terms, how it works and why is it widely used in mission critical systems deployments. - We will understand the 'magic' behind Aerospike ability to handle small, medium and even Petabyte scale data, and still guarantee predictable performance of sub-millisecond latency - We will learn how Aerospike devops is different than other solutions in the market, and see how easy it is to run it on cloud environments as well as on premise. We will also run a demo - showing a live example of the performance and self-healing technologies the database have to offer.

Photos (17)