Yes, there is a difference. The second sentence is correct.
Come (actually come with) is used to talk about joining a movement of the speaker's / hearer's even if go is used for the movement itself.
We are going to the theatre tonight. Would you like to come with us? (NOT Would you like to go with us?)