Computers vs Humans in Understanding Language

1. Computers vs Humans in Understanding Language Christos Christodoulopoulos Amazon Research Cambridge UCL 12.03.19

2. Computers vs Humans in Understanding Language Christos Christodoulopoulos Amazon Research Cambridge UCL 12.03.19

3. Computers’ Side: Alexa • “Alexa, how tall is the Gherkin?” The Gherkin's height is 591 feet. • “Alexa, how tall is a gherkin?” The average height of a pickled cucumber is 2.3 inches. • “Alexa, who was the US president when the Queen was a teenager?” Franklin D. Roosevelt and Harry S. Truman were the US Presidents when Elizabeth II was a teenager.

4. Computers’ Side: Deep Learning • Games • (super)human level performance on Chess, Go, Atari 2600 games • Image Recognition • human level performance on object detection (ImageNet) • Natural Language • Speech recognition: (near) human level performance on certain domains • Speech synthesis: (near) human level naturalness • Question answering: (super)human level performance on SQuAD • Machine translation: much better, but not human-level; hard to tell

5. Computers’ Side: Are we there yet? • What is Language Understanding? • How can we measure it? • Turing Test • Loebner Prize • Alexa Prize • Winograd Schema Challenge • Adversarial Examples [example from Athalye et al. 2018]

6. Computers’ Side: Are we there yet? • What is Language Understanding? • How can we measure it? • Turing Test • Loebner Prize • Alexa Prize • Winograd Schema Challenge • Adversarial Examples “Peyton Manning became the first quarterback ever to lead two different teams to multiple Super Bowls. He is also the oldest quarterback ever to play in a Super Bowl at age 39. The past record was held by John Elway, who led the Broncos to victory in Super Bowl XXXIII at age 38 and is currently Denver’s Executive Vice President of Football Operations and General Manager” “What is the name of the quarterback who was 38 in Super Bowl XXXIII?” Prediction: John Elway [example from Jia and Liang, 2017]

7. Computers’ Side: Are we there yet? • What is Language Understanding? • How can we measure it? • Turing Test • Loebner Prize • Alexa Prize • Winograd Schema Challenge • Adversarial Examples “Peyton Manning became the first quarterback ever to lead two different teams to multiple Super Bowls. He is also the oldest quarterback ever to play in a Super Bowl at age 39. The past record was held by John Elway, who led the Broncos to victory in Super Bowl XXXIII at age 38 and is currently Denver’s Executive Vice President of Football Operations and General Manager. Quarterback Jeff Dean had jersey number 37 in Champ Bowl XXXIV.” “What is the name of the quarterback who was 38 in Super Bowl XXXIII?” Prediction: Jeff Dean [example from Jia and Liang, 2017]

8. Vector models • One of the biggest wins of recent NLP • Input to (almost) every NN system • Continuous representation of words • Similarity models across levels • characters, tokens, sentences, documents • Out Of Vocabulary words • Multi-modal (e.g. VQA) • Cross-lingual alignments

9. Machine language acquisition Infant vs. machine language acquisition - an extension of the innate vs. ‘blank slate’ debate? • Machine learning cannot happen without biases • Biases reveal regularities in data • Can we create biases that reflect our current theories? • Spoilers: yes! (see last slide)

10. Affective speech: Understanding Should machines detect affect in human voices? Progress • Deep learning models from raw data (e.g. Trigeorgis et al. 2016) • Applications e.g. ASD assessment (Ram and Ponnusamy, 2016) Challenges • What is the impact of misclassification? • Cross-cultural adaptation

11. Affective speech: Producing Should computers attempt to produce emotional speech? Progress • Not as advanced as Speech Emotional Recognition • Tahon et al., 2018: “emotion-specific pronunciations are too subtle to be perceived by testers” Challenges • Uncanny valley effect • Moore, 2016: “a speech-enabled robot should have a robot voice” • What is a false positive/negative in responding?

12. Embodiment Does the machine need to be embodied and placed in a social setting (like a human) to learn a language? • Solving the circularity of definitions • Eat -> To consume (something) by putting it into the mouth and swallowing it • Consume -> To eat

13. Embodiment Does the machine need to be embodied and placed in a social setting (like a human) to learn a language? • Can we get away without embodiment? • Depends on how we measure understanding e.g. for metaphors [source: dandelion.eu]

14. Embodiment Does the machine need to be embodied and placed in a social setting (like a human) to learn a language? • Littleto no work on social/cultural embodiment

15. Birds, airplanes, and language Can a machine solving a problem (e.g. learning/understanding language) one way tell us anything about how humans might solve the same problem? • If airplanes don’t have to flap their wings, why do machines have to understand language the same way humans do? • Bird and airplanes take advantage of physical laws • Language is a human, social construct

16. Bridging the gap Computational models as testbeds of (psycho)linguistic theories • Testing Syntactic Bootstrapping • Incremental syntactic parsers • Cognitive load prediction • Eye-gaze for NN attention • Multi-task learning for linguistic representations • Neurocomputational models