Combining Human and Machine Intelligence for NLP

Data trumps algorithms for Natural Language Processing. Where the right machine learning algorithm can increase accuracy by 5%, annotating more training data often gives 20% or greater improvement. This can be the difference between 70% and 90% accuracy, and for many of Idibon’s clients this is the difference between unusable and actionable information. Rob will go over how Idibon approaches the combination of human and machine intelligence. Once humans are in the loop, they become the most expensive part of the equation. Increasing the accuracy efficiently is the complex co-optimization of both machine learning and analyst interaction, taking into account factors like: data coverage; the attention and agency of the analysts; the interfaces that allow analysts to annotate and supply better data; and strategies to resolve inter-analyst disagreement.