By far the most popular algorithm for Spaced Repetition is SM2. Sites like Anki, Duolingo, Memrise, and Wanikani all chose SM2 over later SM-iterations (eg. SM15) because it is extremely simple, yet effective. Despite that, it still has a number of glaring issues. In this article I explain those issues, and provide a simple way to resolve them.
Original SM2 Algorithm
The algorithm determines which items the user must review every day. To do that, it attaches the following data to every review-item:
- easiness:float – A number ≥ 1.3 representing how easy the item is, with 1.3 being the hardest. Defaults to 2.5
- consecutiveCorrectAnswers:int – How many times in a row the user has correctly answered this item
- nextDueDate:datetime – The next time this item needs to be reviewed
When the user wants to review, every item past its nextDueDate is queued. After the user reviews an item, they (or the software) give themselves a performanceRating on a scale from 0-5 (0=worst, 5=best). Set a cutoff for an answer being “correct” (defaults to 3). Then make the following changes to that item:
easiness += 
consecutiveCorrectAnswers = 
nextDueDate = 
Problems with SM2
SM2 has a number of issues that limit its usefulness/effectiveness:
Problem: Non-generic variable ranges
The variable ranges are very specific to the original software, Supermemo. easiness is a number ≥ 1.3, while performanceRating is an integer from [0,5]
Solution
Normalize easiness and performanceRating to [0,1].
This requires setting a max value for easiness, which I set to 3.0. I also replaced easiness with difficulty, because it’s the more natural thing to measure.
Problem: Too many items per day
Because every day we do all the overdue items, it’s easy to encounter situations where one day you have only 5 items to review, and the next you have 50.
Solution
Only require doing the items that are the most overdue. Use “percentage” of due-date, rather than number of days, so that 3 days overdue is severe if the review cooldown was 1 day, but not severe if it was 6 months.
This allows review sessions to be small and quick. If the user has time, they can do multiple review sessions in a row. As a bonus, this allows “almost overdue” items to be reviewed, if the user has time.
Problem: Overdue items all treated equally
If the user completes a month-overdue item correctly, it’s likely they know it pretty well, so showing it again in 6 days is not helpful. They should get a bonus for correctly getting such overdue items correct.
Additionally, the above problem/solution allows almost-overdue items to be reviewed. These items should not be given full credit.
Solution
Weight the changes in due-date and difficulty by the item’s percentage overdue.
Problem: Items learned together are reviewed together
Items that are learned together and always correctly answered will always be reviewed together at the same time, in the same order. This hinders learning if they’re related.
Solution
Add a small amount of randomness to the algorithm.
Other adjustments
- The quadratic term in the difficulty equation is so small it can be replaced with a simpler linear equation without adversely affecting the algorithm
- Anki, Memrise, and others prefer an initial 3 days between due-dates, instead of 6. I’ve adjusted the equations to use that preference.
The Modified “SM2+” Algorithm
Here is the new algorithm, with all the above improvements in place.
For each review-item, store the following data:
- difficulty:float – How difficult the item is, from [0.0, 1.0]. Defaults to 0.3 (if the software has no way of determining a better default for an item)
- daysBetweenReviews:float – How many days should occur between review attempts for this item
- dateLastReviewed:datetime – The last time this item was reviewed
When the user wants to review items, choose the top 10~20 items, ordered descending by percentageOverdue (defined below), discarding items reviewed in the past 8 or so hours.
After an item is attempted, choose a performanceRating from [0.0, 1.0], with 1.0 being the best. Set a cutoff point for the answer being “correct” (default is 0.6). Then set
percentOverdue = 
difficulty += ![percentOverdue*\frac{1}{17}(8-9*performanceRating)\text{, clamp to [0,1]}](https://www.blueraja.com/blog/wp-content/ql-cache/quicklatex.com-1a2f2b592e3fad4d2d9e04f536f33b2c_l3.png "Rendered by QuickLaTeX.com")
difficultyWeight = 
daysBetweenReviews *= 
Daily Review Sessions
The above algorithm determines which items to review, but how should you handle the actual review?
That’s the topic for my next post – stay tuned!
29 Comments
Thanks for sharing
Congratulations.
I’m waiting for the next post.
Do you know any already implemented in any programming language (PHP, Python, Java) ?
I dont get your formula behind percentoverdue.
If card is correct, it is at least 2, and probably higher
If card is incorrect, it is set to 1.
shouldnt it be the other way round, caused items to be shown are the top entries when it is ordered descending by percentoverdue.
Or I am just missing something.
When you are initially ordering the items, there is no “correct” or “incorrect” yet.
I meant for you to order by the “correct” definition, though looking back I realize that’s unclear. Hmm.
2 is the maximum, not the minimum.
I get it now.
I sort of misread the Min, thing.
Makes sense.
Is there way to implement SM2 algorithm and get minutes too along with dates? like Anki does it?
are you sure memrise app use sm2 algorithm?
This is very easy to read. Love it. Are you planning to have a second part to this blog soon? Thanks.
It would be really awesome to see part two of this. It’s such a great resource on a little discussed topic and well worked out.
You probably have a typo here:
> easiness += -0.8 + 0.28*performanceRating + 0.02*performanceRating^2
in the original (https://www.supermemo.com/english/ol/sm2.htm) it’s:
> – 0.02*performanceRating^2
I wonder where did you get the
> 6 * (easiness ** (consecutiveCorrectAnswers – 1))
formula? I’m only seeing
> I(n):=I(n-1)*EF
in the original.
Great approach. I wonder where did you get the nextDueDate formula as well.
> 6 * (easiness ** (consecutiveCorrectAnswers – 1))
Great post!
Question: have you tested this algorithm? What were your conclusions? Unclear whether this is purely theoretical or not.
I’m a little confused about how daysBetweenReviews is calculated. It looks like that value can never be greater than 5 with the default values used in this article. How do you get longer cooldown periods, like the 6 months you referenced when you suggested ordering cards to be studied by percent overdue ?
1 + (difficultyWeight – 1) * percentOverdue. If we take the max value for difficultyWeight (3) and the max value for percentOverdue (2) the result is 1+ (3 – 1) * 2 = 5
looks like I missed the “*=”. I read it as “=”. Thanks for the article!
Waiting for part 2
question for the SM2+ algo
According to the formula, all words that the user gives a performance Rate value < 0.6 (say "failed-words"), will have a value 1 for percentOverdue.
Then how can the algorithm decide which words to select from these failed-words in next round?
Thank you for sharing. I am doing a project that requires using SM2. I will appreciate that you could explain this “The above algorithm determines which items to review, but how should you handle the actual review?”, because I have been waiting for your next post but can’t see any signs yet.
I would love to see the second post, too! So far I see a couple JavaScript implementations up and the start of one in PHP. I’m working on a Python one for my own uses. One little thing I notice looking at this is that a float number of days might be awkward to work with. I’ll probably use minutes or seconds in mine.
Another detail I noticed while implementing this. You have a variable called “percent_overdue”, but it should really be called something like “waiting_progress” or something. It represents the ratio of the time since the last review and the time that should go by since the last review. If it is less than 1, then the item should not be reviewed. Come to think of it, maybe you were meaning to write about this in your next post 😉
Hello,
When I create a card, how much should I set its initial DaysBetweenReviews?
Hi Blueraja!
I really need your help with Daily Review Sessions. Daily Review Sessions is very important because without it users will have to learn a huge number of items every session. Please help me!
and Also, I like your blog for your electricity post. It helped me with my science homework:)
Are you okay? I don’t think a follow-up is coming?
I implemented a far less obscure version of SM2, to help people actually understand this.
https://gist.github.com/doctorpangloss/13ab29abd087dc1927475e560f876797
It would be great if you clarified where you got the consecutive correct answers param & next due date calculation because that’s not part of the original SM2 algorithm and it’s just adding to the confusion
Love the thinking, has anyone actually tried this?
Does SM 2 base its algorithm on forgetting curve equations? if not, then wouldn’t that be the way to go?
Hi,
I am very inspired with your spaced repetition logic. I have calculated the variables but getting confused to use it in actual algorithm.
You mentioned that your next blog will be related to use of these variables in algorithm but I am unable to find it.
Could you please guide me in this regard.
Thanks,
Vijay
can you add the code to git hub and share it
Trackbacks/Pingbacks