The construction 'accuse someone of going' is, of course, grammatically possible, but would have quite the wrong meaning: the speaker would be accusing you of having physically gone somewhere in order to quit. It would have no future sense whatsoever.