OWASP Toronto | Inference Servers: new technology, same old security flaws.

** The event is hosted at 111 Peter St Suite 804, Toronto, ON M5V 2H1 **

TALK
-----------

Inference Servers: new technology, same old security flaws.

Summary:

AI and LLM based applications are taking the industry by storm. While a lot time is spent on evaluating prompt injection, there is an entire ecosystem of applications that allow models to be run and used. These applications have their own important security considerations that you may not be aware of.
Inference Servers are used to host machine learning models and expose APIs that allow other components to perform inference on those models. These servers often expose additional APIs that allow users to load new models. Often, this can be abused to perform remote code execution. While this technology is new, the baseline security configurations for many of these products are a relic from the past.
In this talk we’ll learn about what an inference server is, how they work, and how you can achieve remote code execution in them. This talk is mainly focused on the practical security risks involved in this ecosystem. Finally, I will share details about a couple of CVEs related to TorchServe

Presenters:

Pratik Amin has been an Application Security practitioner for over 15 years. He currently works as a Principal Security Consultant at Kroll (previously Security Compass Advisory). In this role, he spends most of his time performing AppSec pentests and digging into interesting technology.