Language evolution seminar

Speaker: Rahma Chaabouni (Facebook AI Research)

Title: Compositionality and Generalization in Emergent Languages

Abstract: Natural language allows us to refer to novel composite concepts by combining expressions denoting their parts according to systematic rules, a property known as compositionality. Linguists agree that compositionality plays a crucial role in natural language, accounting for its productivity (i.e., the ability to express an infinite number of ideas by finite means).

Despite the importance of productivity, deep neural networks, known for their impressive NLP performances, often fail to generalize to unseen examples. In this talk, I ask whether the language emerging in deep agents possesses a similar ability to refer to novel primitive combinations and whether it accomplishes this by strategies akin to intuitive compositionality. I address this by first looking at the generalization performances of deep agents on unseen inputs. Our experiments show that given sufficiently large input spaces, the emergent language will naturally develop the ability to refer to novel composite concepts. Second, I introduce different measures of intuitive compositionality that the community is looking at. Based on these measures, I show experiments demonstrating no correlation between the degree of compositionality of an emergent language and its ability to generalize. I conclude the talk by suggesting that, while compositionality is not necessary for generalization, it provides an advantage in terms of language transmission: The more compositional a language is, the more easily it will be picked up by new learners.

Contact

Seminars are organised by the Centre for Language Evolution

Henry Conklin

Centre for Language Evolution