Name: Minions, a powerful framework to handle complex operational tasks in Pinot
Start: 2024-01-24T18:00:00-08:00
End: 2024-01-24T19:30:00-08:00

**Join us for this exclusive meetup to learn about Minions- a powerful framework to handle complex operational tasks in Pinot .**

Apache Pinot is a real-time distributed OLAP datastore that is designed to support high-throughput queries with low latency for various analytics use cases. To maintain data integrity, result accuracy, and system efficiency, Pinot performs background operational tasks such as data compaction, GDPR data purging, table repartitioning, and schema evolution reindexing. These tasks can be resource-intensive and potentially impact query performance if executed on the same component responsible for query execution.

To address this challenge, Pinot utilizes Minion, which is built upon Apache Helix's task framework. Minion handles computationally intensive operational tasks, effectively offloading these workloads from the query execution component. This separation ensures that operational tasks do not compromise query performance. Minion is designed to be easily extensible and pluggable, serving not only to address performance issues but also to create data ingestion and backfilling pipelines, saving time for operators who would otherwise need to build custom solutions.
The talk delves into the Minion framework and certain Infrastructure components, providing a exploration of how it is leveraged in various operational tasks, showcasing its versatility and effectiveness in maintaining the optimal performance of Apache Pinot

[Manish S](https://www.linkedin.com/in/manishswaminathan/) is a Software Engineer at [StarTree](www.startree.ai)
\-\-\-\-\-

For more content, events and opportunities, **[Join the community](https://dev.startree.ai/slack-invite)**

StarTree

Real-Time Analytics with Apache Pinot™ by StarTree

StarTree Community

Technology

Open Source

Software Development

Web Technology

Big Data

Data Analytics

OLAP

Distributed Systems

Real-Time

Minions, a powerful framework to handle complex operational tasks in Pinot 

Online event

Share

Real-Time Analytics with Apache Pinot™ by StarTree

Minions, a powerful framework to handle complex operational tasks in Pinot

Real-Time Analytics with Apache Pinot™ by StarTree

Details

Members are also interested in