Dimensionality Reduction: PCA, SVD, & NMF -- what they can and cannot do for you

Dr. Scott Schwartz is an instructor in the Galvanize Data Science Immersive program at Galvanize SoHo location in downtown New York City. Dr. Schwartz received a BS and BA summa cum laude in Computer Science and Mathematics from Trinity University, and his PhD in Statistics at Duke University. After a postdoctoral fellowship in the program for bioinformatics and nutrition and Texas A&M University, he worked for several years at the Texas A&M AgriLife next generation sequencing core supporting both academic and agribusiness research programs, and then moved to the University of Texas to work on biofuels crops for two years before joining galvanizes instructional staff in Austin, Texas, in early 2016. Dr. Schwartz has since moved to Galvanizes NYC campus.

Going all the way back to the mixture model and Dirichlet process based clustering used in his dissertation, Dr. Schwartz has utilized unsupervised learning techniques in his work. In the biology and genetics problem domains he worked in (association studies, gene networks, and genetic mapping) — so-called “large p small n” problems — Dr. Schwartz was increasingly confronted by the problems of “multiple testing” and the “curse of dimensionality”, and unsupervised learning techniques were often leveraged in these contexts in response to these issues. One interesting sounding, and seemingly relevant unsupervised learning approach in this regard is “dimensionality reduction.” In this talk, the dimensionality reduction methodologies of PCA, SVD, and NMF are presented and their capabilities and limitations are discussed. Unfortunately, these techniques are not actually designed to address the “multiple testing” problem and the “curse of dimensionality” per se, but they do nonetheless provide a very powerful and interesting set of methodologies that have proven to be very useful in a number of problem domains.