Jenkins meets Spark: building a continuous integration intelligence

Analyzing Jenkins data with Spark and Zeppelin

Jenkins is widely used for Continuous Integration and Continous Delivery across a lot of companies and it is pretty easy to build CI/CD workflows with it. However, detailed analysis of workflow execution results is not always easy to achieve and visualise in a convenient form. This talk presents some ideas on how to utilize Spark for collection and transformation of Jenkins logs and to perform interactive analysis and visualization with Zeppelin and Highcharts as a frontend. We will have a look into what kind of insights can be extracted from Jenkins test reports, how to provide faster feedbacks on test results and how to prioritize tests execution based on this data.

Speaker bio:
Rockie Yang is a passionate software developer currently working for Ericsson. He has designed software architecture for embedded system, inter process communication, multi tenant queue system. He enjoys working between business and implementation, eager to see business cases flawlessly transformed implementation with simplicity in mind. He recently working on HDFS, Spark, Zeppelin, and Highcharts and released zeppellin-highcharts to add great visualization library Highcharts to Spark ecosystem.

Agenda:
5.30 lead-in by organizer
5.45 part 1: Crawling data from Jenkins
6.30 break: beers and pizza
6.50 part2: Analyzing Jenkins data with Spark and Zeppelin