Name: Chunked Text Processing and Record Linkage
Start: 2018-05-30T19:00:00+02:00
End: 2018-05-30T21:00:00+02:00
Location: Amsterdam Public Library (OBA)

This time we will have two speakers from the Dutch Statistics Office CBS.

1) Chunked, dplyr based processing for text files
Edwin de Jonge, https://github.com/edwindj

R is a great tool, but processing large text files with data is cumbersome. chunked helps you to process large text files with dplyr while loading only a part of the data in memory. It builds on the execellent R package LaF Processing commands are writing in dplyr syntax, and chunked (using LaF) will take care that chunk by chunk is processed, taking far less memory than otherwise. chunked is useful for selecting columns, mutating columns and filtering rows.

2) reclin: a toolkit for record linkage and deduplication
Jan van der Laan, https://github.com/djvanderlaan

Record linkage, entity resolution, data matching. All terms for determining which records belong to the same entity or object. When all records are located in the same dataset this is also called deduplication. When a unique identifier is available that is also registered without errors, this is simple. However, often one has to work with name, address and date fields that contain errors such as misspellings. The reclin package provides tools to help with this and implements one of the most used methods: probabilistic record linkage. I will try to explain the general methodology of record linkage and show how reclin can be used.

Hannes Mühleisen

David Jankoski

amst-R-dam

Technology

Programming in R

Machine Learning

Predictive Analytics

R Project for Statistical Computing

Chunked Text Processing and Record Linkage

Amsterdam Public Library (OBA)

Share

amst-R-dam

Chunked Text Processing and Record Linkage

amst-R-dam

Details

Members are also interested in