Teaching Machines to Code: Neural Markup Generation

We present a neural transducer model with visual attention that learns to generate LATEX markup of a real-world math formula given its image. Applying sequence modeling and transduction techniques that have been very successful across modal-ities such as natural language, image, handwriting, speech and audio; we construct an image-to-markup model that learns to produce syntactically and semantically correct LATEX markup code over 150 words long and achieves a BLEU score of 89%; improving upon the previous state-of-art for the Im2Latex problem. We also demonstrate with heat-map visualization how attention helps in interpreting the model and can pinpoint (localize) symbols on the image accurately despite having been trained without any bounding box data.

Speaker Bio:
Sumeet Singh is Founder and CTO of Untrix, and former Technical Director of Symantec. He is a software technologist and deep learning modeler/researcher with over two decades experience in areas of Cyber Security, Data Platforms, Cyber Communications and AI. He has a Masters degree in Systems Science and Automation from the Indian Institute of Science.
https://untrix.github.io/i2l

Agenda:
6:30–7 pm Meet and greet
7-8 pm Presentation and discussion
8-8:30pm Social and catch up