Nextflow: Data-Science Pipelines with Dr. Rob Syme

This is a past event

26 people went

The Perth Data Science Meetup
The Perth Data Science Meetup
Public group

CORE/FLUX Offices, Parmelia House

Lower Floor, Parmelia House, 191 St George's Terrace, · Perth

How to find us

Down one level from the Lobby of Parmelia House (191 St George's Terrace, corner of Mill Street)

Location image of event venue

Details

If you're lucky, your data science workloads involve interacting with well structured data and writing custom tools with one language. The unfortunate reality for many is a regular battle with long pipelines of binaries and polyglot scripts that all read and write files.

Nextflow is a fluent DSL modelled around the UNIX pipe concept that simplifies writing parallel and scalable pipelines in a portable manner. We'll discuss the ways Nextflow saves you from drowning in the accumulated digital detritus and what sets it apart from the galaxy of other excellent workflow engines available (GNU Make, Rake, Taverna, Luigi, etc).

If you're interested in writing jobs in your preferred language(s) in a way that makes them easy to migrate from your laptop to the cloud or HPC environments, Nextflow is free, open source, docker-friendly, and worth a look.

---------

Rob is a research associate at the Centre for Crop and Disease Management (CCDM) at Curtin University. He is looking at the genomes of fungi and the crops they infect to understand the genetic determinants of a handful of plant diseases. He promises to keep his presentation focused on data and not to bore you with talk of fungi and genomes.