Apache NiFi + Hacking around the IoTree (Meetup #39)

Christmas edition with a special speaker: Look forward to a talk on data flow management tool Apache NiFi by Andy LoPresto (Cloudera), a hackathon ("aka community programming") with Apache PLC4X, NiFi & more, mulled wine & Christmas cookies!
https://nifi.apache.org/

In cooperation with Hadoop & Spark User Group Rhein-Main (HSUG Rhein-Main): https://www.meetup.com/de-DE/HSUG-Rhein-Main/

Talk

BYOP: Custom Processor Development with Apache NiFi

Description
Apache NiFi, a robust, scalable, and secure tool for data flow management, ships with over 286 processors to ingest, route, manipulate, and exfil data from a variety of sources and consumers. Many users require proprietary protocol parsing, running inside connected cars, or offloading massive hardware metrics from oil rigs in the most remote environments.

Abstract
Apache NiFi, a robust, scalable, and secure tool for data flow management, ships with over 286 processors to ingest, route, manipulate, and exfil data from a variety of sources and consumers. But many users turn to NiFi to meet unusual requirements — from proprietary protocol parsing, to running inside connected cars, to offloading massive hardware metrics from oil rigs in the most remote environments. Rather than posting a community request for custom development or offloading unusual demands to unnecessary external systems, there’s an answer in NiFi. Learn how NiFi allows you to quickly prototype custom processors in the scripting language of your choice against live production data without affecting your existing flows. Easily translate prototypes to full-fledged processors to optimize performance and leverage the full provenance reporting infrastructure. Discover how the framework provides conventions to streamline your development and minimize common boilerplate code, and the robust testing framework to make testing easy, and dare we say, fun.

Expected prior knowledge / intended audience
Developers and data flow managers should have passing knowledge of Apache NiFi as a platform for routing, transforming, and delivering data through systems (a brief overview will be provided). The intended audience will have experience with programming in Java, Groovy, Ruby, Jython, ECMAScript/Javascript, or Lua.

Takeaways
Attendees will gain an understanding in writing custom processors for Apache NiFi, including the component lifecycle, unit and integration testing, quick prototyping using a scripting language of their choice, and the artifact publishing and deployment process.

Speaker
Andy LoPresto has been at Cloudera (previously Hortonworks) since 2015, where he currently leads the Data In Motion Engineering Security Team. In this role he serves as both a Committer and Project Management Committee (PMC) Member for Apache NiFi, an open source, robust, secure data routing and delivery system. Andy focuses on security concerns within NiFi including identity management, TLS negotiation, data protection, access control, encryption and hashing, but also contributes across the entirety of the project, including custom processor development, Industrial Internet of Things (IIot) integration, and healthcare & financial use case application. Andy is also involved with the sub-projects -- Apache MiNiFi, which drives edge data collection, including secure command and control and immediate data provenance and governance; and Apache NiFi Registry, which provides complementary applications for flow versioning, development to production lifecycle, asset management, and extension library services. Over the last few years, he has presented about NiFi and MiNiFi in Seoul, Vilnius, Barcelona, Singapore, Tokyo, Melbourne, Berlin, Sydney, Brussels, and the US.