In AmE, "Let's go out" means the same thing as "Let's go out together" in BrE (and in AmE too, I assume). It's an invitation to a romantic date.
BrE speakers mean completely different things by:
1) "Would you like to go out?"
2) "Would you like to go out with me?"
It appears that in AmE, 1) also means the same as 2).
Incidentally, the traditional polite term for a man and a woman seeing each other romantically was that they were 'walking out'. For example: "I see John is walking out with Jane these days." And you thought you were confused before .
Back to the question...
I would say: "Fancy a walk?"
Could our American friends comment on how that would sound?