Using LSTMs to Assess the Obligatoriness of Phonological Distinctive Features for Phonotactic Learning

Nicole Mirea and Klinton Bicknell

Northwestern University

Abstract

To ascertain the importance of phonetic information in the form of phonological distinctive features for the purpose of segment-level phonotactic acquisition, we compare the performance of two recurrent neural network models of phonotactic learning: one that has access to distinctive features at the start of the learning process, and one that does not. Our work interrogates the assumption that phonological distinctive features be specified in advance of learning through the following questions:

Is external information regarding phonological distinctive features a necessary prerequisite for learning phonotactic patterns at the segment level?
Must models become sensitive to phonological properties of incoming segments in order to represent phonotactic generalizations?

To answer them, we utilize recurrent neural networks with long short-term memory (LSTM) nodes, which have shown considerable success in learning patterns at both the word (Sundermeyer et al., 2015) and character levels (Kim et al., 2016).

1 Introduction

Knowing a language involves having systematic expectations about the sequential sound patterns within syllables and words in the language. This sensitivity helps language users segment a continuous stream of speech (Vitevitch et al., 1997), incorporate new words into the lexicon (Storkel et al., 2006), and reconstruct parts of an utterance that may have been obscured by noise.

Phonotactic patterns are typically stated in terms of generalizations over natural classes. If phonetic segments must be specified in terms of distinctive features in advance of phonotactic learning, we would expect a model that encodes phonetic segments as vectors of features to ultimately represent phonotactic generalizations more accurately than one that initializes segments randomly. If, on the other hand, the features are not obligatory, the feature-naive model should have no advantage.

2 Related Work

Experimental work supports a characterization of phonotactics as gradient expectations over sound sequences (Frisch et al., 2000). Many models have addressed phonotactic learning but presuppose a set of hierarchical distinctive features predetermined by researchers. Our research questions this assumption and investigates its implications for phonotactic learning models.

3 Model

Our models are recurrent neural networks with LSTM nodes. Their task is to incrementally predict the next phonetic segment in a sequence. The structure includes:

3.1 Input Layer

Each phonetic segment is read as a one-hot vector, with the number of nodes equal to the phonetic inventory size.

3.3 Recurrent Layers

Each recurrent layer consists of 512 LSTM nodes, allowing for better representation of complex dependencies.

3.4 Output Layer

This layer outputs a probability distribution over the next segment using a softmax function to normalize the predictions.

4 Experiment 1: Evaluating on a Held-Out Test Set

We investigate whether pre-specified distinctive features improve the likelihood assigned to a held-out test set. Two conditions were implemented: feature-aware (specification based on distinctive features) and feature-naive (random initialization).

4.1 Method

All models were trained on a subset of the CELEX2 corpus (Baayen et al., 1995), adjusting weights via backpropagation to minimize cross-entropy loss.

5 Experiment 2: Comparison to Human Judgments

We evaluated our models using non-words designed by Daland et al. (2011) and correlated model log-likelihoods with human-derived judgments on wordlikeness ratings.

6 Clustering of Learned Phone Embeddings

We performed qualitative cluster analysis of phonetic segment embeddings learned by models. Comparing clustering metrics indicates that the feature-aware model maintains manner-based distinctions, while the feature-naive model captures phonetic information differently.

8 Implications

Our results suggest that phonological distinctive features are not necessary for phonotactic acquisition.

9 Conclusion

Phonotactic acquisition can occur without prior knowledge of distinctive features, indicating a need to reevaluate models relying on predetermined feature sets.