AWS Data Pipelines 2023

Join us to explore and discover the power of data pipelines in AWS.

Learn to consolidate data from all your disparate sources into one common destination. Data pipelines enable near real time decision making and reliable business insights from consistent high quality data. In addition, you will discover low cost storage solutions for later playback and deeper data analysis.

During this session, we will dive into the fundamental concepts behind data pipelines, serverless computing, scalability, cost-effectiveness and developer experience. Real-world examples and use cases will help illustrate how data pipelines can help you in your business.

Speakers will share practical insights, best practices, and key considerations for selecting the right approach.

Whether you're a developer, architect, or simply curious about cloud computing, and data pipelines, this talk will provide valuable insights.

Presentation Outline

• Overview of data pipelines
• Architecture
• Raw zone data ingestion
• Streaming data transformation using Kinesis Data Analytics
• Using Kinesis Firehose to feed services such as Athena and OpenSearch
• Use Cases
• Raw Zone Storage
• Data Enrichment
• OpenSearch
• Amazon Redshift Serverless streaming ingestion
• Athena
• Demo
• Source Code

6:00 - 6:15 pm: Mix and mingle
6:15 - 6:30 pm: Announcements and intro
6:30 - 7:30 pm: Main talk