[Disclaimer: I am not a teacher, but am a native speaker and hold an undergraduate degree in English.}
Back to Wikipedia:
English plural - Wikipedia, the free encyclopedia
(scroll down to "almost-regular plurals.")
This seems to be one of those rules based on pronunciation. "Rooves" indeed used to be the plural of roof, but is now considered archaic.
I am sitting here reciting the plural of various voiceless-fricative ending words and find I am perfectly capable of saying calf(s) and wolf(s), so maybe in another hundred years the rest of these V's will disappear. In the meantime, I suppose you are going to have to memorize.
PS: The dwarf/dwarves note is
fascinating.