It sounds unnatural to me. I can understand people being afraid of certain weather phenomena (tornadoes, for example), but I've never heard of anyone being afraid of the cold. They might not like it and they might feel uncomfortable but is it scary? I don't think so. Of course, if someone were heading somewhere which has abnormally low temperatures, the idea of being in those temperatures might scare them.
Retired English Teacher