Natural Language Generation and defect segments detection


Details
Agenda:
18:00-18:30: Gathering and networking
18:30-19:00: Factuality and Grounding in Natural Language Generation, by Erez Schwartz
19:00-19:30: Inferring Implicit Relations in Complex Questions with Language Models, by Uri Katz
19:30-20:00: Detecting defect segments by the Laplace smoothing, by Uri Itai
Factuality and Grounding in Natural Language Generation, by Erez Schwartz
The state of the art in Natural Language Processing (NLP) is characterized by large models trained on large amounts of data. These models have achieved impressive results, but have also raised concerns about the lack of grounding and factuality in the resulting models. This lecture will discuss the factuality and grounding issues in large Language models, and how to approach solving them.
Erez is a Data Scientist and Algorithm Engineer, specializing in NLP and AI. He has a degree in Electrical Engineering from the Technion, and has worked as a Data Scientist in the defence sector. Erez currently works as an algorithm engineer in AI21 Labs, where he is working on developing Large Language Models alongside task-specific models.
Inferring Implicit Relations in Complex Questions with Language Models, by Uri Katz
A prominent challenge for modern language understanding systems is the ability to answer implicit reasoning questions, where the required reasoning steps for answering the question are not mentioned in the text explicitly. In this work, we investigate why current models struggle with implicit reasoning question answering (QA) tasks, by decoupling inference of reasoning steps from their execution. We define a new task of implicit relation inference and construct a benchmark, IMPLICITRELATIONS, where given a question, a model should output a list of concept-relation pairs, where the relations describe the implicit reasoning steps required for answering the question. Using IMPLICITRELATIONS, we evaluate models from the GPT-3 family and find that, while these models struggle on the implicit reasoning QA task, they often succeed at inferring implicit relations. This suggests that the challenge in implicit reasoning questions does not stem from the need to plan a reasoning strategy alone, but to do it while also retrieving and reasoning over relevant information.
Uri Katz is a Computer Science PhD candidate at Bar-Ilan University, focusing on Natural Language Processing, Information Retrieval, and Reasoning. He is advised by Prof Yoav Goldberg. Before starting his PhD, Uri completed his M.Sc at Tel Aviv University under Prof Jonathan Berant working on Question answering and Commonsense capbilities of large language models
Detecting defect segments by the Laplace smoothing, by Uri Itai
Defect segmentation in logs is challenging due to the two main constraints. The first is that the defect segment has a high failure rate. Otherwise, it won’t be a defect segment. The second issue is the size of the segment. Small segments are more prone to a high failure rate. However, this does not necessarily mean much. In addition, there is no business motivation to tackle small segments. The naive approach is taking hard thresholds and namely, focusing on segments by hard criteria. This path can lead to a combinatorial explosion of threshold setting. Moreover, there might be edge cases that make this approach to be very messy.
To overcome this we decided to take a more Bayesian approach. In particular, Laplace smoothing. This allows us to detect, with statistical significance, the defective segments. After achieving this, we generalize this technique to multi-class and categorical features.
Applying the additive average allows us to tackle continuous variables and regression problems. In this talk, we will state the problem. Then define the Laplace smoothing. Following this, we will explain how to generalize this technique to multi-class and regression problems. we end by showing additional uses for this technique. I.e., Bayesian trees.
Uri Itai has a PhD in applied mathematics from the Technion (IIT) under the supervision of Prof. Nira Dyn. His dissertation was the study of constructing surfaces for non-Euclidean manifolds.
After graduating he was working in various fields of data science, algorithmic trading, cancer research and more

Natural Language Generation and defect segments detection