High-Performance Computing with R (Think 1024 cores, 4 TB RAM)

Ever wondered what it's like to play with R on 1024 cores and 4 TB of shared memory? We were curious and are lucky enough to have Amy Szczepanski talk at our next useR meeting. Amy works with R and High-Performance Computing at the University of Tennessee and will join us Thursday 28 July 2011 at 6:15pm (This will replace our August meeting). All levels are welcome and encouraged to attend. Bring your questions!

Agenda:

Pizza and networking (BYOB) Amy Szczepanski's Presentation (Abstract Below) Got a related topic? Email ideas to Josh at josh@rstudio.org

Title: R with High Performance Computing: Parallel processing and large memory

Amy Szczepanski - Remote Data Analysis and Visualization Center, University of Tennessee

As data sets get larger and analyses become more sophisticated, laptop and desktop computers can no longer handle the demands for processor time and memory. Carrying out these larger analyses often requires migrating to high performance computing (HPC). There are both key differences and noteworthy similarities between HPC systems and traditional desktop computers. This talk will explain why understanding these architectures is important when moving one's R code to an HPC system and will give an overview of the R packages that can be helpful for achieving parallelism on various architectures. Furthermore, as R is a memory bound language, we will discuss the limitations of R for working with large datasets and some of the packages that have been developed to work around these limitations. Finally, we will introduce the computational resources managed by XSEDE for the NSF and explain how researchers can apply for allocations of time on these systems.

The Remote Data Analysis and Visualization Center ( http://rdav.nics.tennessee.edu (http://rdav.nics.tennessee.edu/)) operates Nautilus, an SGI Altix UV 1000 with 1024 cores and 4 TB shared memory, in support of NSF's mission for open science research. Nautilus is an XD visualization and data analysis resource, with allocations managed by XSEDE ( http://www.xsede.org (http://www.xsede.org/)).