Alane Suhr

Learning and Using Natural Language in Interaction

Learning to use language in interactions by using language in interactions

I will discuss grounded natural language instruction-following in collaborative interactions. First, I will describe CerealBar, a new environment and task for studying grounded natural language interaction. In the CerealBar game, two players, a leader and a follower, work together to collect sets of matching cards by moving through a 3D environment. The game incentivizes collaboration through natural language instruction. I will describe an automated system which follows human-written instructions in CerealBar. Second, I will discuss recent work that investigates the kind of language people use in CerealBar through the lifetime of a community of players, with the surprising result that as the community ages, instructions written by leaders become more complex. Finally, I will discuss an upcoming TACL paper where we use interaction with human followers to train a system that generates instructions in CerealBar.

Alane is a final-year PhD student at Cornell University (based in NYC at Cornell Tech) advised by Yoav Artzi. Alane’s interests are language grounding, including language in interactions, semantic parsing, and visual reasoning.

Presentation Materials

Talk Video