Building Efficient Web Scrapers: Rust vs. Python for Data Ingestion

Join us for our next Lunch & Learn 💫, where we’ll dive into building efficient web scrapers with Rust and Python. Camille Owens will walk us through the fundamentals of web scraping, key differences from APIs, and best practices for performance, error handling, and production deployment.

Agenda

13:00 - 13:05
Welcome & Introduction
Caroline Morton

13:05 - 13:45
Building Efficient Web Scrapers: Rust vs. Python for Data Ingestion
Camille Owens
Web scraping is a powerful technique for extracting data from websites, but how does it compare to using APIs? In this Lunch and Learn, I will dive into web scraping fundamentals, covering key differences between scrapers and API calls. I will explore Rust and Python libraries like reqwest, scraper, and Polars for data extraction, along with best practices for asynchronous HTTP requests, error handling, and parsing HTML. Will also cover performance trade-offs, cost considerations, and DevOps insights when deploying scrapers in production. Whether you are building data pipelines or exploring new tools, this session will provide valuable insights into efficient data ingestion strategies.

--

FAQs
Are there any House Rules?
At SurrealDB, we are committed to providing live and online events that are safe and enjoyable for all attending. Please review our website for more information: https://surrealdb.com/legal/code-of-conduct

Is Women in Rust only for women?
We set up Women in Rust to foster a more inclusive and diverse Rust community by providing a dedicated space for women to connect, learn, and grow. Our online and in-person events are designed to support this mission. However, we also want to contribute to the wider Rust community, which is why all of our talks will be available to watch on YouTube Youtube after the event. We also welcome you to use our Notion Resources page, where you can find useful articles, reading materials and advice to help you on your journey in learning Rust.

How can I get involved with Women in Rust?
We’re always excited to welcome new contributors! You can get involved by:

• Speaking at our events: Share your expertise or experience with the community.
• Hosting online workshops: Lead interactive sessions to help others learn and grow.
• Spreading the word: Help us reach more people by sharing our events and initiatives.

Simply fill out this form to express your interest: https://surrealdb.typeform.com/to/B2gk7bzz. We’ll be in touch to explore how we can collaborate.Let’s grow together—your involvement makes a difference!