Learning from Omission

Bill McDowell

Duolingo
Pittsburgh, PA 15206
mcdowell@duolingo.com

Noah D. Goodman

Stanford University
Stanford, CA 94305
goodman@stanford.edu

Abstract

Pragmatic reasoning allows humans to go beyond the literal meaning when interpreting language in context. Previous work has shown that such reasoning can improve the performance of already-trained language understanding systems. Here, we explore whether pragmatic reasoning during training can improve the quality of learned meanings. Our experiments on reference game data show that end-to-end pragmatic training produces more accurate utterance interpretation models, especially when data is sparse and language is complex.

1 Introduction

From cooperative reference games. As a motivating case, consider an instance of the color reference task from Monroe et al. (2017)—shown in the first row of Table 1. In this task, a speaker communicates a target color to a listener in a context containing two distractor colors; the listener picks out the target based on what the speaker says. In the first instance from Table 1, the speaker utters “dark blue” to describe the target. Whereas “dark” and “blue” also apply to the target, they lose their informativity in the presence of the distractors, and so the speaker pragmatically opts for “dark blue”.

A listener who is learning the language from such examples might draw several inferences from the speaker’s utterance. First, under the assumption that the speaker is informative, a “literal” learner might infer that “dark blue” applies to the target shade more than the distractors. Second, a “pragmatic” learner might consider the cheaper alternatives—“dark” and “blue”—that have occurred in the presence of the same target in prior contexts, and infer that these alternative utterances must also apply to the distractors given the speaker’s failure to use them. The pragmatic learner might thus gain more semantic knowledge from the same training instances than the literal learner: pragmatic reasoning can reduce the data complexity of learning.

In light of these arguments, we leverage the pragmatic inference described here in training neural network models to play reference games. For formal, probabilistic representations of contextual reasoning in our training objectives, we embed neural language models within pragmatic listener and speaker distributions, as specified by the Rational Speech Acts (RSA) framework (Goodman and Frank, 2016; Frank and Goodman, 2012). Pragmatic inference allows our models to learn from indirect pragmatic evidence of the sort described above, yielding better calibrated, context-sensitive models and more efficient use of the training data.

We compare pragmatic and non-pragmatic models at training and at test, while varying conditions on the training data to test hypotheses regarding the utility of pragmatic inference for learning.

Table 1: Speaker utterances describing (1) colors and (2) color grids to differentiate them from distractors. A learner might draw inferences about fine-grained linguistic distinctions by explaining the speaker’s failure to use cheaper alternatives in context (e.g. they might infer that “blue” and “dark” apply to some distractors in 1). These inferences have the potential to increase in number and in strength as dimensionality of the referents and utterance complexity increase (as in 2).

2 Related Work

Prior work has shown that neural network models trained to capture the meanings of utterances can be improved using pragmatic reasoning at test time via the RSA framework (Andreas and Klein, 2016; Monroe et al., 2017; Goodman and Frank, 2016; Frank and Goodman, 2012). For instance, Monroe et al. (2017) train context-agnostic (i.e. non-pragmatic) neural network models to learn the meanings of color utterances using a corpus of examples of the form shown in the first line of Table 1. At evaluation, they add an RSA layer on top of the trained model to draw pragmatic, context-sensitive inferences about intended color referents. Other related work explores additional approaches to create context-aware models that generate color descriptions (Meo et al., 2014), image captions (Vedantam et al., 2017), spatial references (Golland et al., 2010), and utterances in simple reference games (Andreas and Klein, 2016). Each of these shows that adding pragmatics at test time improves performance on tasks where context is relevant.

3 Approach

We compare neural nets trained pragmatically and non-pragmatically on a new color-grid reference game corpus as well as the color reference corpus from Monroe et al. (2017). In this section, we describe our tasks and models.

In both reference games, our listener models reason about a round r represented by a single training/testing example of the form (O;U;t) where O is the set of objects observed in the round (colors or color-grids), U is a sequence of utterances produced by the speaker about the target (represented as a token sequence), and t is the target index in O. The models predict the most likely referent O_t of an utterance within a context O according to an RSA listener distribution l(t | U;O) over targets given the utterances and a context. In pragmatic models, a nested structure allows the listener to form its beliefs about the intended referent by reasoning recursively about speaker intentions with respect to a hypothetical “literal” (non-pragmatic) listener’s interpretations of utterances. This recursive reasoning allows listener models to account for the speaker’s context-sensitive, pragmatic adjustments to the semantic content of utterances.

4 Experiments
We investigate the value of pragmatic training by estimating the parameters in the RSA “literal meaning” function L for l1 (pragmatic) and l0 (non-pragmatic) distributions according to the maximal likelihood of the training data for the color and grid reference tasks. We then evaluate meanings L from each training procedure using pragmatic l1 inference (and non-pragmatic l0 inference, for completeness). We perform this comparison repeatedly to evaluate the value of pragmatics at training and test under various data conditions. In particular, we evaluate the hypotheses that (1) the pragmatic inferences enabled by the l1 training will reduce sample complexity, leading to more accurate meaning functions especially under small data sizes, and (2) the effectiveness of the l1 training over l0 training will increase on a more difficult reference game task containing higher-dimensional objects and utterances—i.e. pragmatic training will help more in the grids task than in the colors task.

| Data | Color Accuracy | Grid Accuracy | | --- | --- | | Full | 0.9003 | 0.9318 | | Close | 0.8333 | 0.9024 | | Split | 0.8970 | 0.9291 | | Far | 0.9696 | 0.9642 |

5 Conclusion

Our experiments provide evidence that using pragmatic reasoning during training can yield improved neural semantics models. This was true in the existing color reference corpus, where we achieved state-of-the art results, and even more so in the new color-grid corpus. We thus found that pragmatic training is more effective when data is relatively sparse and the domain yields complex, high-cost utterances and low-cost omissions over which pragmatic inferences might proceed. Future work should provide further exploration of the data regime in which pragmatic learning is most beneficial and its correspondence to real-world language use.