Re: other way to say "I am not willing to do that"
A lot depends on what has been proposed.
Did the person think you'd like it, but you don't? Oh thank you, but I'd rather not.
Did the person make a suggestion that he realized was not something you would want to do, that causes and advantage for him and a disadvantage to you? No, I'm sorry, that's out of the question.
Was it a true impertinence? The Queen of England herself could ask you to pole dance naked in Trafalgar Square, and you'd be within your rights to say "I should say NOT!"
I'm not a teacher, but I write for a living. Please don't ask me about 2nd conditionals, but I'm a safe bet for what reads well in (American) English.