The verb want requires an object. That object can be a noun (or a noun phrase), "I don't want candy," or a verb (in its infinitive form), "I don't want to go." In your sentence the object of want is the verb move. Since the verb has already been stated, only its infinitive marker, to is necessary.
My home is remote and a far cry from a public phone box but I won't move and anyway I don't want to [move].
Interested in Language