Human & Linguistic Factors: The 8th NLP Dublin Meetup

IMPORTANT: Please fill out name and surname in this form (https://goo.gl/forms/7paWx3h2miKLpOTq2) to be allowed entry by the building security.

We are thrilled to announce our next NLP Dublin meetup! This time, we are overjoyed to be hosted for the second time by Workday at their offices in Kings Building, May Lane, Smithfield.

https://secure.meetupstatic.com/photos/event/9/a/a/7/600_462399591.jpeg

For our eighth meetup, we are delighted to have an all-female speaker panel consisting of three amazing speakers: Dr. Sheila Castilho, Dr. Carla Parra Escartín, and Dr. Teresa Lynn.

Past editions of our meetup have particularly focused on machine learning applications in Natural Language Processing. However, the results of such approaches are meaningless if they do not provide tangible benefits to their end-users, us. Furthermore, in order to apply such approaches to different languages, we need to understand the underlying linguistic universals.

In this meetup, we will thus look at how we can evaluate models from a human perspective. Sheila Castilho will first give a general overview of human evaluation. Carla Escartín will then provide a case study of human-computer interaction for machine translation post-editing. Finally, Teresa Lynn will present the Universal Dependency Project (http://universaldependencies.org/), which allows enables cross-lingual parsing by capturing linguistic universals across languages.

AGENDA:

[18:00 - 18:30] Registration, pizza, and networking

[18:30 - 19:00] Dr. Sheila Castilho (https://www.linkedin.com/in/sheila-castilho-0550b030/?locale=de_DE) (Postdoc, ADAPT Centre, DCU): "Human Evaluation - Why do we need it?"

Abstract: In both research and practice evaluation is a complex task. In the machine translation field, evaluation involves a range of linguistic and extra linguistic factors. Evaluation is important since it enables to report the progress in the development of NLP systems. In this talk, we show the main practices for (human) evaluation of machine translation and how some of these practices may be used in the other NLP areas.

Short bio: Sheila Castilho holds a Master in Natural Language Processing and a PhD from Dublin City University. Currently, she is a post-doctoral researcher at the TraMOOC project at ADAPT Centre focusing on machine and human evaluation of automatically translated subtitles and user-generated content.

[19:00- 19:30] Dr. Carla Parra Escartín (https://sites.google.com/site/carlaparraescartin/) (Marie Skłodowska-Curie fellow, ADAPT centre, DCU) : "Living on the edge: productivity gain thresholds in machine translation evaluation metrics"

Abstract: Machine Translation (MT) has become a reality in the translation industry. Over the past few years, translators have experienced the introduction of Machine Translation Post-Editing (MTPE) tasks in their workflows, and are constantly asked for discounts for MTPE tasks. However, the question of whether MT output has a positive impact in productivity is still open.

In this talk, I will first present how professional translators work and how MT is introduced in real production workflows. Then, I will present an experiment involving 10 professional translators in which we measured their productivity when translating from scratch, post-editing Translation Memory fuzzy matches and post-editing Machine Translation output. I will discuss the results of our experiment, and the productivity thresholds that we identified.

Short bio: Carla Parra Escartín obtained her PhD in Computational Linguistics at the University of Bergen (Norway) and is currently a Marie Skłodowska-Curie fellow at the ADAPT Centre in DCU.

[19:30- 20:00] Dr. Teresa Lynn (https://www.linkedin.com/in/teresa-lynn-41430817/) (Postdoc, ADAPT Centre, DCU) : "Universal Dependencies Project"

Abstract: Syntactic parsing is a valuable component of a number of language technology applications. These days, parse features are often used in data-driven development of applications such as grammatical error correction, machine translation, question-answering, dialogue systems, sentiment analysis, summarization, information retrieval etc.

In particular, statistical dependency parsing plays an important role in cross-lingual and multilingual processing. Until recently, however, progress in the area of cross-lingual processing was hampered by variations in the annotations and design of treebanks that are used to train these parsers. The Universal Dependency Project (http://universaldependencies.org/) (2015) aimed to address these issues by developing universal annotation guidelines that would capture linguistic universals across languages. This enabled the development of a set of harmonized treebanks that lend themselves easily to cross-lingual analysis or processing. The latest release (v 2.0 March 2017) consists of 70 treebanks, covering 50 languages.

In this talk I will provide a 101 introduction to statistical dependency parsing before discussing the UD project and my involvement in it through the development of the Irish Universal Dependency Treebank.

Short bio: Teresa Lynn is a Post-Doctoral Researcher at the ADAPT Centre (http://adaptcentre.ie/), Dublin City University. She completed her PhD through a cotutelle agreement between Macquarie University Sydney (Australia) and Dublin City University (Ireland). She built the first dependency treebank for Irish and trained the first statistical dependency parsing models for Irish.

[20:00 - 21:00] Networking

IMPORTANT NOTES:

• If you can't make it, please RSVP to "NO" as soon as possible so that other people can take your place.

• If you are doing innovative research in NLP or are applying NLP to exciting applications and would like to give a talk, reach out to us! Similarly, if you are interested in sponsoring or hosting this event, please contact us.

Co-organized by Aylien (http://aylien.com/)

https://secure.meetupstatic.com/photos/event/9/a/b/7/600_462399607.jpeg