AI Builders and Learners SF

Welcome to AI Build & Learn, a weekly AI engineering stream where we pick a new topic and learn by building together.

​This event is about generating images with AI. There's no single model we're locked into, the point is to explore what's out there and actually try a few. We'll focus on open-source models, but you're welcome to bring commercial ones (DALL·E, Midjourney, and friends) if you want to compare.

​We'll try to look at both major families: diffusion-based models (the dominant approach today) and transformer-based / autoregressive image models. I'll research and try some of the best open-source options ahead of the stream, and if time allows we'll also try fine-tuning with LoRA to teach a model a new style or subject.

​Some things to look up to get started:

​Open-source models:

• ​FLUX.2 (Black Forest Labs): current open-weight quality benchmark; DiT backbone, up to 4MP, multi-image reference
• ​Stable Diffusion 3.5 / SDXL (Stability AI): deepest ecosystem — LoRA, ControlNet, inpainting, countless fine-tunes
• ​Qwen-Image (Alibaba): strong all-rounder, especially good at rendering text (including Chinese)
• ​Z-Image / Z-Image-Turbo: Apache-2.0, near FLUX.2 quality in just a few steps (fast and commercial-friendly)
• ​HunyuanImage (Tencent): another high-quality open-weight option

​Tooling:

​Most agents use tools by emitting one structured tool call at a time. The model picks a tool, waits for the result, picks the next, and every intermediate result flows back through its context. As the number of tools and steps grows, that gets expensive, slow, and brittle.

​Code mode flips this. Instead of calling tools one at a time, the agent writes code that calls them as functions, runs that code in a sandbox, and returns only what matters back to the model. Why this works well in practice: models are trained on far more code than tool-call traces, so they compose operations more reliably in a language they already know. Loops, branching, filtering, and data transforms happen in the sandbox instead of as separate round-trips. Large intermediate results stay out of the context window, and tool definitions load on demand instead of all up front. The result is an agent that does more per step and spends fewer tokens doing it.

​The obvious worry is letting a model write and run code. We'll handle that with Flyte's sandboxed orchestrator: the generated code can do pure control flow like loops, branching, and wiring tool calls together, but has no access to the filesystem, network, or OS. The actual tool or MCP calls get dispatched to isolated container tasks. You get the flexibility of letting a model write code without handing it the keys to the machine.
​In this hands-on workshop, we'll build a code mode agent that writes and executes code to call tools, then deploy it behind a simple UI. The whole pipeline runs on Flyte 2/Union, so runs are durable and reproducible, steps are cached, and the same code scales from a laptop to a multi-node cluster without rewrites.
​By the end, you'll have a working code mode agent and a reusable pattern you can point at your own tools.

​What we'll cover

• ​Why direct tool-calling breaks down at scale, and what code mode changes
• ​Running model-generated code safely in Flyte's sandboxed orchestrator, with heavy work dispatched to isolated tasks
• ​Orchestrating with Flyte 2: cached steps, durable runs, and execution that scales
• ​Deploying the agent with a UI, with a path to production

​What you'll leave with

• ​A working code mode agent that writes and runs code to use tools
• ​A reusable pattern you can adapt to your own tools and MCP servers
• ​A clear sense of when code mode beats direct tool-calling, and when it does not

Welcome to AI Build & Learn, a weekly AI engineering stream where we pick a new topic and learn by building together.

​This event is about generating video with AI, picking up where the image generation event left off. As with images, there's no single model we're committed to — we'll explore several and see how far the open-source options have come. Open source is the focus, but commercial models (Sora, Runway, and others) are fair game if you want to compare.

​Most current video models build on the same diffusion foundations as image generation, extended across time and increasingly using transformer/DiT backbones. We'll try a few open-source text-to-video and image-to-video models and talk through the practical tradeoffs: quality, clip length, speed, and hardware requirements.

​Some things to look up to get started:
​
Open-source models:

• ​Wan 2.2 (Alibaba Tongyi): versatile MoE model: text-to-video, image-to-video, and editing in one
• ​HunyuanVideo (Tencent): strong cinematic / photorealistic quality
• ​LTX-Video / LTX-2 (Lightricks): the fast one; ~5s clip in under a minute on a single GPU
• ​CogVideoX (Zhipu / THUDM): best at following detailed, multi-part prompts
• ​Mochi 1 (Genmo): flow-matching model known for fluid, coherent motion
• ​Stable Video Diffusion (Stability AI): earlier image-to-video, still widely used

​Tooling:

July's book is "RAG with Python Cookbook"!

This is a casual-style event. Not a structured presentation on topics. Sometimes, the discussion even drifts away from the chapters, but feel free to grab the mic to help steer it back.

Feel free to join the discussion even if you have not read the book chapters! :)