DataFrame - a Swiss Army Knife of Java Data Processing with Andrus Adamchik

As Java developers, we do a lot of data processing. If you have terabytes pumped through your system daily, maybe you would reach for Spark, Flink or another “big data” solution. But there are also many everyday programming tasks that do not warrant the overhead and complexity of traditional data pipelines: do an ad hoc analysis of app logs, clean up and persist an uploaded Excel file, run a simple ETL copying data between two tables in different databases, etc. So, can you use the “big data” techniques without big data infrastructure?
This talk will focus on “DataFrame” - an in-memory data structure that is a 2D table with various operations, like filtering, column and row transformations, joins, aggregations, etc. I will use an open source DFLib library (https://dflib.org) and Jupyter notebook to demonstrate how to do data processing in any Java app without much fuss.