DataTalks #36 @ DLD: NLP Models in the Medical Domain

Our 36th DataTalks meetup will be hosted at Gugy's Toy Factory, as part of the DLD TLV events!🎉

** Find more info about DLD TLV here: https://www.dldtelaviv.com/2023/https://www.dldtelaviv.com/2023/

We will focus on interesting implementations and usage of NLP methods in biological and medical domains!

Location: Rothschild Blvd 39, Tel-Aviv
🚘Note: No parking is available! The closest parking lot is at חניון בית ציון. Rothschild Blvd 41, Tel Aviv-Yafo

𝗔𝗴𝗲𝗻𝗱𝗮:
🍕 19:00 - 19:15 - Mingling, etc.
🔶 19:15 - 20:00 – NLP for non-language purposes — Applying natural language processing to medical EHR data
🦭 20:00 - 20:10 - Short break
🔷 20:10 - 20:55 – ProteinBERT: A universal deep-learning model of protein sequence and function

***

Talks #1: NLP for non-language purposes — Applying natural language processing to medical EHR data

Speaker: Tal Geller, Data Science Team Lead at Diagnostic Robotics.

Abstract: Electronic medical records include information about a patient’s health history, such as diagnoses, medicines, tests, treatment plans and more. As part of these records, health providers use various coding systems to record the services a patient received. Modeling these codes as words and sentences allows us to apply Natural Language Processing techniques to extract information about patients' healthcare trajectories and apply additional Machine Learning methods to create predictions and recommendations about health outcomes and treatment.

***

Talks #2: ProteinBERT: A universal deep-learning model of protein sequence and function

Speaker: Dan Ofer, PhD student at Hebrew University, Senior Data Scientist at Medtronic.

Abstract: Self-supervised deep language modeling has shown unprecedented success across natural language tasks, and has recently been repurposed to biological sequences. However, existing models and pretraining methods are designed and optimized for text analysis. We introduce ProteinBERT, a deep language model specifically designed for proteins. Our pretraining scheme combines language modeling with a novel task of Gene Ontology (GO) annotation prediction. We introduce novel architectural elements that make the model highly efficient and flexible to long sequences. The architecture of ProteinBERT consists of both local and global representations, allowing end-to-end processing of these types of inputs and outputs. ProteinBERT obtains near state-of-the-art performance, and sometimes exceeds it, on multiple benchmarks covering diverse protein properties (including protein structure, post-translational modifications and biophysical attributes), despite using a far smaller and faster model than competing deep-learning methods. Overall, ProteinBERT provides an efficient framework for rapidly training protein predictors, even with limited labeled data.

***

The DLD Tel Aviv Innovation Festival is well established in today's era of invention and imagination. This celebration of knowledge is the event that companies, startups, investors, entrepreneurs, and others excitedly anticipate for a glimpse into the future of technology and its international ecosystem.