A Trainable Spaced Repetition Model for Language Learning

Burr Settles
Duolingo
Pittsburgh, PA USA
burr@duolingo.com

Brendan Meeder
Uber Advanced Technologies Center
Pittsburgh, PA USA
bmeeder@cs.cmu.edu

Abstract

We present half-life regression (HLR), a novel model for spaced repetition practice with applications to second language acquisition. HLR combines psycholinguistic theory with modern machine learning techniques, indirectly estimating the "halflife" of a word or concept in a student's long-term memory. We use data from Duolingo a popular online language learning application to fit HLR models, reducing error by 45%+ compared to several baselines at predicting student recall rates. HLR model weights also shed light on which linguistic concepts are systematically challenging for second language learners. Finally, HLR was able to improve Duolingo daily student engagement by 12% in an operational user study.

1 Introduction

The spacing effect is the observation that people tend to remember things more effectively if they use spaced repetition practice (short study periods spread out over time) as opposed to massed practice (i.e., "cramming"). The phenomenon was first documented by Ebbinghaus (1885), using himself as a subject in several experiments to memorize verbal utterances. In one study, after a day of cramming he could accurately recite 12-syllable sequences (of gibberish, apparently). However, he could achieve comparable results with half as many practices spread out over three days.

The lag effect (Melton, 1970) is the related observation that people learn even better if the spacing between practices gradually increases. For example, a learning schedule might begin with review sessions a few seconds apart, then minutes.

The effects of spacing and lag are well-established in second language acquisition research (Atkinson, 1972; Bloom and Shuell, 1981; Cepeda et al., 2006; Pavlik Jr and Anderson, 2008), and benefits have also been shown for gymnastics, baseball pitching, video games, and many other skills. See Ruth (1928), Dempster (1989), and Donovan and Radosevich (1999) for thorough meta-analyses spanning several decades.

Most practical algorithms for spaced repetition are simple functions with a few hand-picked parameters. This is reasonable, since they were largely developed during the 1960s-80s, when people would have had to manage practice schedules without the aid of computers. However, the recent popularity of large-scale online learning software makes it possible to collect vast amounts of parallel student data, which can be used to empirically train richer statistical models.

In this work, we propose half-life regression (HLR) as a trainable spaced repetition algorithm, marrying psycholinguistically-inspired models of memory with modern machine learning techniques. We apply this model to real student learning data from Duolingo, a popular language learning app, and use it to improve its large-scale, operational, personalized learning system.

2 Duolingo

Duolingo is a free, award-winning, online language learning platform. Since launching in 2012 more than 150 million students from all over the world have enrolled in a Duolingo course, either via the website or mobile apps for Android, iOS, and Windows devices. For comparison, that is more than the total number of students in U.S. elementary and secondary schools combined. At least 80 language courses are currently available or under development for the Duolingo platform. The most popular courses are for learning English, Spanish, French, and German, although there are also courses for minority languages (Irish Gaelic), and even constructed languages (Esperanto).

More than half of Duolingo students live in developing countries, where Internet access has more than tripled in the past three years (ITU and UNESCO, 2015). The majority of these students are using Duolingo to learn English, which can significantly improve their job prospects and quality of life (Pinon and Haydon, 2010).

2.1 System Overview

Duolingo uses a playfully illustrated, gamified design that combines point-reward incentives with implicit instruction (DeKeyser, 2008), mastery learning (Block et al., 1971), explanations (Fahy, 2004), and other best practices. Early research suggests that 34 hours of Duolingo is equivalent to a full semester of university-level Spanish instruction (Vesselinov and Grego, 2012).

2.2 Spaced Repetition and Practice

Once a lesson is completed, all the target words being taught in the lesson are added to the student model. This model captures what the student has learned, and estimates how well she can recall this knowledge at any given time. Spaced repetition is a key component of the student model: over time, the strength of a skill will decay in the student's long-term memory, and this model helps the student manage her practice schedule.

Duolingo uses strength meters to visualize the student model. These meters represent the average probability that the student can, at any moment, correctly recall a random target word from the lessons in this skill. At four bars, the skill is "golden" and considered fresh in the student's memory. At fewer bars, the skill has grown stale and may need practice.

3 Spaced Repetition Models

In this section, we describe several spaced repetition algorithms that might be incorporated into our student model. We begin with two common, established methods in language learning technology, and then present our half-life regression model which is a generalization of them.

3.1 The Pimsleur Method

Pimsleur (1967) was perhaps the first to make mainstream practical use of the spacing and lag effects, with his audio-based language learning program. He referred to his method as graduated-interval recall, whereby new vocabulary is introduced and then tested at exponentially increasing intervals, interspersed with the introduction or review of other vocabulary. However, this approach is limited since the schedule is pre-recorded and cannot adapt to the learner's actual ability.

3.2 The Leitner System

Leitner (1972) proposed a different spaced repetition algorithm intended for use with flashcards. It is more adaptive than Pimsleur's, since the spacing intervals can increase or decrease depending on student performance.

3.3 Half-Life Regression: A New Approach

We now describe half-life regression (HLR), starting from psychological theory and combining it with modern machine learning techniques.

Central to the theory of memory is the Ebbinghaus model, also known as the forgetting curve. This posits that memory decays exponentially over time:

$$ p = 2 ^ {- \Delta / h}.$$

Figure 4(a) shows a forgetting curve with half-life $ h=1 $.

To illustrate, Figure 4(b) shows a student-word learning trace over the course of a month. Each indicates a data instance: the vertical position is the observed recall rate for each practice session, and the horizontal distance between points is the lag time.

Our goal is to find the best model weights ( \Theta^{*} ) to minimize some loss function:

$$ \Theta^ {*} = \arg \min _ {\Theta} \sum_ {i = 1} ^ {D} \ell \left(\langle p, \Delta , \mathbf {x} \rangle_ {i}; \Theta\right). $$

4 Experiments

In this section, we compare variants of HLR with other spaced repetition algorithms in the context of Duolingo. First, we evaluate methods against historical log data, and analyze trained model weights for insight.

Table 2 presents evaluation results using historical log data. For all three metrics, HLR with lexeme tag features is the best (or second best) approach. These HLR variants are also the only ones with positive correlation.

5 Other Related Work

Just as we drew upon the theories of Ebbinghaus to derive HLR as an empirical spaced repetition model, there has been other recent work drawing on other (but related) theories of memory.

6 Conclusion

We found that lexeme tag features failed to improve predictions much; however, they provide insight into the aspects of language that are more or less challenging for students.

Data and Code

To facilitate research in this area, we have publicly released our data set and code from our experiments at GitHub.

Acknowledgments

Thanks to our collaborators at Duolingo for help gathering data from various parts of the system.