Let’s learn computer vision

Build a Hand-Controlled Painting App with Real-Time Camera Stream Analysis
In this hands-on workshop, we’ll be building an exciting app that recognizes hand movements via a live camera stream and uses them to draw on a digital canvas! Imagine the classic “Paint” program from Windows, but instead of a mouse, you’ll control the painting with your hands moving in mid-air—captured live by your camera.

### What We’ll Do:

• Capture video stream from your camera
• Detect hand positions in real-time
• Transmit hand coordinates to the front-end using Server-Sent Events (SSE)
• Create an interactive HTML canvas for drawing
• Use hand movements to control the drawing on the canvas, replacing traditional mouse input

### Tech We’ll Use:

• Server-Sent Events (SSE):
  Learn how to send live data from the server to the browser.
  SSE in the browser |
  SSE on the server with FastAPI
• HTML Canvas (2D Context):
  Build a drawing interface using the browser’s 2D canvas.
  Canvas API Docs
• Python Generators:
  Efficiently handle data flow in real time using generators.
  Generators in Python
• OpenCV Video Capture:
  Capture video from your webcam using this powerful library.
  OpenCV Official Site
• MediaPipe Hands:
  Track and identify hand positions with precision using Google’s MediaPipe.
  MediaPipe Hands Documentation

### Requirements:

Basic knowledge of JavaScript and Python is recommended. We'll be writing code from scratch to create a fully functional demo. You should be comfortable with:

• Installing Python and npm packages
• Editing code in an IDE

If you're new to some of these technologies, don't worry! You'll be able to follow along, and code will be shared in real-time using HackMD.

After we’ve built the app, we’ll dive into discussions about MediaPipe and compare it with other computer vision frameworks like OpenCV and YOLO.

### Event Schedule:

• 20 min: Video stream capture setup
• 20 min: Hand position detection
• 20 min: Sending hand positions to the browser
• 30 min: Building a classic “Paint” app with mouse control
• 30 min: Switching from mouse to hand-controlled drawing
• 15 min: Switching between drawing modes

### Useful Resources:

• Free 6-hour Advanced Computer Vision Course
• How MediaPipe Hand Tracking Works
• Python refresher: Learn Python in minutes
• JavaScript refresher: Learn JavaScript in minutes

***

Join us for a fun and interactive workshop where you’ll learn about cutting-edge hand-tracking technology and apply it to build a creative, real-time app. Whether you're a seasoned developer or just curious about computer vision, this workshop is sure to inspire!