Locking should prevent this from being a problem, but taking a reference to the
cached iterator and mutating it directly causes occasional crashes for me for
reasons I can't quite figure out.
This fixes the issue and is arguably more sane anyway, so whatever.