PyData Montreal #20

Agenda:
(All times in EST)
6:00 pm — Introduction
6:10 pm — "Small Big Data: using NumPy and Pandas when your data doesn't fit in memory"
6:50 pm — "Zero to Production-Ready: a best-practices process for Docker packaging"

About Itamar:
Itamar helps teams using Python ship faster code, and ship code faster. He has been using Python since 1999, and worked on scientific computing, distributed systems, and more. His open source work includes Fil (https://pythonspeed.com/products/filmemoryprofiler/), a memory profiler data intensive processing, and Eliot (https://eliot.readthedocs.io/en/stable/), the causal logging library.
You can learn more about Python performance, optimizing memory usage, and Python Docker packaging at https://pythonspeed.com.

---------------------------------------------------------------------------------
Talk #1: Small Big Data: using NumPy and Pandas when your data doesn't fit in memory

Your data is too big to fit in memory—loading it crashes your program—but it’s also too small for a complex Big Data cluster. How to process your data simply and quickly? In this talk, you’ll learn the basic techniques for dealing with Small Big Data: money, compression, batching, and indexing. You’ll specifically learn how to apply these techniques to NumPy and Pandas, but you’ll also learn the key concepts you can apply to other libraries and the specifics of your particular data.

---------------------------------------------------------------------------------
Talk #2: Zero to Production-Ready: a best-practices process for Docker packaging

You know the basics of packaging your Python application for Docker, but do you know enough to run that image in production? Bad packaging can result in security and production problems, not to mention wasted time trying to debug unreproducible errors.
And even if you figure out the best practices, there's still a huge number of details to get right, many of which interact with each other in unexpected ways. My personal list includes over 70 Docker packaging best practices, and it keeps growing. So, where do you start? What should you do first?
To help you quickly package your application in a production-ready way, this talk will give you a process to help you prioritize and iteratively implement these best practices by starting with the highest priority best practices (security, automation), moving on the correctness and reproducibility, and finally focusing on optimization.