The Future of Automation: Apache Airflow Meets Generative AI

📣📣 JOIN US FOR TALKS ON GENAI IN AIRFLOW 📣📣

GENAI + AIRFLOW
Data scientists and engineers are continuously discovering innovative applications for Generative AI and Machine Learning within their Airflow environments, and we are excited to share these advancements.

Join us at Amazon's offices in Palo Alto for three compelling presentations exploring the future of Generative AI in Airflow and its current practical applications.

AGENDA

• 5:30 PM-6 PM: Arrivals, Shoe Customizations, Eat, Drink and Network!
• 6 PM PM-7 PM PM: Presentations!
• 7 PM- 8 PM: Shoe Customizations continued, Eat more, drink more, and network!

PRESENTATIONS

Talk #1: Gen AI in Airflow: Today and tomorrow

• Speaker: Vikram Koka, Chief Strategy Officer at Astronomer
• Abstract: Based on user surveys over a quarter of Apache Airflow users are deploying Gen AI workloads into production using Airflow. This is primarily due to the deterministic nature of Airflow which is critical for all data workflows including Gen AI. However, there is always scope for improvement. As part of the upcoming Airflow 3 release, we are working to make Gen AI workflows easier to run. The focus of this talk is on these upcoming changes and the thinking behind them. We would love to get your feedback on the same.

Talk #2: Optimizing GenAI: Customization Strategies for LLMs with Airflow

• Speaker: Vincent La, Principal Data Scientist at Laurel
• Abstract: Laurel offers an AI-driven timekeeping solution for accounting and legal firms, automating timesheet creation by capturing digital work activities. This session covers AI projects like UTBMS Code Prediction for legal bills and Bill Creation using RAG for billable entries. We’ll also discuss using Airflow for daily model retraining, redeployment, and cost management through efficient summarization of activities.

Talk #3: Accelerating Airflow DAG development using Amazon Q, a generative AI-powered assistant

• Speakers: Aneesh Chandra PN, Principal Analytics Specialist at AWS & Sriharsh Adari, Sr Solutions Architect at AWS
• Abstract: In this talk, discover how you can leverage workspace context awareness in Amazon Q Developer to accelerate tedious and time-consuming activities such as DAG refactoring for Airflow version or package upgrades, implementing best practices with DAG development and recommend entire functions in real-time, directly from your editor. The session will equip you with techniques that allow you to spend less time on boilerplate and repetitive code patterns, and more time on what truly matters: building exceptional orchestration workflows.