Local optima in classical feedforward neural networks and what to do about them.

Dmitriy Golovashkin will be presenting: "Local optima in classical feedforward neural networks and what to do about them."

About the talk:

Feedforward neural network is a popular data mining technique, which can be thought of as mapping, capable to learn intricate nonlinear relationships between input and output datasets.

Generally speaking what is characteristic to most neural network models is that they are formulated as nonconvex optimization problems. Nonconvexity implies that there may be (and very often are) multiple local minima, multiple different solutions. The difference between a local minimum and the global one can be significant, therefore special care is needed to "escape" local extrema.

Occasionally, commercial and open source neural network packages resort to restarting nonlinear optimization at random initial points in a hope to get closer to the global minimum. However the chances of capturing the global optimum even on medium/small size datasets are often negligibly small.

In this talk we will present several MIP based deterministic global optimization techniques,(techniques which guarantee global optimality), and discuss their R implementation.

Talk Agenda:

• Introduction to neural networks
• Present an overview of several popular R ( http://www.r-project.org ) neural network packages: nnet, neuralnet, RSNNS
• Discuss deterministic global optimization techniques & R based implementation
• Demos
• Discussion / Questions

Speaker Info:

Dmitry Golovashkin, PhD

Oracle, Advanced Analytics

http://www.oracle.com/us/products/database/options/advanced-analytics/overview/index.html

Lightning Talks: We are looking for individuals to present lightning talks! If you're interested in giving a quick 5-10 minute presentation, please contact me.

Meetup Schedule:

6:30-6:45 -- socializing

6:45-7:00 -- lightning talks (Contact me if you are interested in presenting)

7:00-8:00 -- main presentation

8:00-8:30 -- socializing

Special Thanks:

Pivotal Labs (http://pivotallabs.com/) -- for hosting!