Python talks with Kiwi.com #2

Join us for this next great meetup in Porto which is part of the Python Tour organized by Kiwi.com.

We’ve prepared two talks for you, one of them also chosen for the next Europython conference. We will also have some fun games and, of course, drinks and networking.

AGENDA

• 19:00 — 19:10. Opening talk
• 19:10 — 19:30. Peter Strečanský "Don't let your service go bankrupt: Pay the technical debt!
• 19:30 — 20:00. Coffee break
• 20:00 — 20:30. Petr Stehlík "The dos and don’ts of task queues"
• 20:30 — 21:30. Networking time

DON'T LET YOUR SERVICE GO BANKRUPT. PAY THE TECHNICAL DEBT

Technical debt (TD) is often considered as a negative concept in software development. When monitored and controlled, it can provide short-term benefits for the company, for example in the form of an earlier release date. When the debt starts to pile up and is not repaid, the development velocity lowers and the development itself becomes more challenging. Much like in the real world, constant indebtedness without repayments is not sustainable in the long run and can lead to eventual bankruptcy.

In this talk, I'll show you how our service went bankrupt, what consequences it caused and how we managed to dig ourselves out of it. I'll go through multiple TD evaluation techniques and show the state of TD in Python open source libraries.

THE DOS AND DON'TS OF TASK QUEUES

Lessons learned about task queues and the final solution of ours in Celery

At Kiwi.com we heavily rely on task queues and asynchronous execution of code to process large amounts of requests coming to our back-ends. With the separation of our codebase to microservices, we can quickly try new tools and different approaches to process these large volumes of requests. The microservice we’ll be talking about is making unreliable slow 3rd party services reliable and asynchronous with a bit of business logic sprinkled on top of it. We’ll tell a failure story of ours but resulting in a valuable lesson.

Most of our services use Celery and it’s the go-to tool for new services as well but we wanted to be different with this new microservice. RQ is the next best choice for task queues and it is presented as simpler and more straightforward than Celery. That can definitely be true but after 3 weeks of research, development and struggling we found out the unpleasant truth about being simple and making the right choices. We won’t talk about comparing the frameworks but rather about the approach on how to experiment with new things in your environment. After that, we’ll present our current setup which can take upon any number of tasks. How we orchestrate the app and continuously integrate and deploy and what fun things await ahead of us in the development.