addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwchatcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrosseditemptyheartexportfacebookfolderfullheartglobegmailgoogleimageimagesinstagramlinklocation-pinmagnifying-glassmailminusmoremuplabelShape 3 + Rectangle 1outlookpersonplusprice-ribbonImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruseryahoo

SF: PL/R and PivotalR in MPP databases, future of PivotalR w/ Hai Qian

PL/R and PivotalR in MPP databases, future of PivotalR w/ Hai Qian

6:00-6:30 pm Food and networking 
6:30-8:00 pm Talk and Q&A 
8:00-8:30 pm Wind down 

In recent years, Big Data has become an important research topic and a very realistic problem in the industry. The amount of data that we need to process is exploding, and the ability of analyzing big data has become the key factor in competition. Big data sets do not fit into a computer’s memory and it would be really slow if the big data sets were processed sequentially. On the other hand, most contributed packages of R are still strictly sequential, single machine, and  are restricted to small data sets that can be loaded into memory. As computing shifts irreversibly to parallel architectures and big data and many big data sets are stored in databases, there is a risk for the R community to become irrelevant.

Here we will introduce how to run machine learning algorithms on these data sets using R.

PL/R is an extension to Postgres databases that enables the user to send any R scripts directly into the database to execute. It has some limitations, which are overcome by PivotalR. PivotalR is a R package that provides a front-end to PostgreSQL and all PostgreSQL-like databases or HDFS. PivotalR also provides the R wrapper for MADlib, an open-source library for scalable in-database analytics. It provides data-parallel implementations of mathematical, statistical, and machine-learning algorithms for structured and unstructured data. Thus PivotalR also enables the user to apply machine learning algorithms onto big data. PivotalR adds functionality that does not exist in MADlib right now, for example, the support for categorical variables.

Hai Qian, a senior software engineer in Pivotal Inc's Predictive Analytics Team working on the development of the in-database machine learning library MADlib.


****SF Giants have a home game today @ 12:45 pm ending approximately 3:45 pm and may cause traffic to be more congested than usual****

Join or login to comment.

Our Sponsors

  • Pivotal

    Pivotal has sponsored our venues, food, beverages, snacks, and video.

  • Uber

    Uber has provided venues, snacks and video

  • eBay

    eBay sponsors venues, and food.

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy