[In-person meetup] NLP Quality and Legal Prompting

Join the 60th NLP Zurich tech shindig back in person!

Florian Cäsar (Founder@Alpaka Ventures) and Dietrich Trautmann (Applied Research Scientist@Thompson Reuters Labs) will give talks respectively on the state of the art in NLP quality evaluation, and legal prompt engineering for multilingual legal judgement prediction.

This event is in-person only and will be followed by a networking apéro. There will be no live-stream. We are looking forward to seeing you all in person!

Agenda:

18:30 Florian Cäsar (Founder@Alpaka Ventures): NLP Quality Matters
18:55 Dietrich Trautmann (Applied Research Scientist@Thompson Reuters Labs): Legal Prompt Engineering for Multilingual Legal Judgement Prediction
19:20 Networking apéro ⊂(◉‿◉)つ

Talk Summary:

NLP Quality Matters
---
NLP is eating the world, but we barely understand how to make its most exciting use cases work. Based on extensive research and interviews with NLP researchers and practitioners, Florian will present the state of quality matters in applied NLP and emerging best practices for making hard NLP tractable.

• Where is the edge of the possible in NLP today?
• How do we know if a model is good? How do we evaluate properly?
• What does quality mean? When does NLP quality matter?
• What can you do today to take NLP quality more seriously?

Legal Prompt Engineering for Multilingual Legal Judgement Prediction
---
Legal Prompt Engineering (LPE) or Legal Prompting is a process to guide and assist a large language model (LLM) with performing a natural legal language processing (NLLP) skill.

Our goal is to use LPE with LLMs over long legal documents for the Legal Judgement Prediction (LJP) task. We investigate the performance of zero-shot LPE for given facts in case-texts from the European Court of Human Rights (in English) and the Federal Supreme Court of Switzerland (in German, French and Italian).

Our results show that zero-shot LPE is better compared to the baselines, but it still falls short compared to current state of the art supervised approaches.

Nevertheless, the results are important, since there was 1) no explicit domain-specific data used - so we show that the transfer to the legal domain is possible for general-purpose LLMs, and 2) the LLMs where directly applied without any further training or fine-tuning - which in turn saves immensely in terms of additional computational costs.

About the speakers:

Florian Cäsar is an NLP practitioner, software engineer and startup founder who most recently built and sold a fintech NLP startup. Currently he is deeply interested in engineering the magic out of powerful deep ML models. Before his NLP startup, Florian created an educational machine learning framework, published an indie puzzle game and researched distributed consensus algorithms.

Dietrich Trautmann is an Applied Research Scientist at Thomson Reuters Labs, Zug, CH. His current work involves the application of Natural Language Processing (NLP) for the legal domain. He contributes to task definitions, the data sets creation processes, improving the labeling quality with automated methods and developing, training, and evaluating of corresponding machine learning models (mostly based on Transformers).