(Not a Teacher)
In your example sentence, there's no difference in meaning, but I wouldn't apply that categorically. For instance:
"He went to the casino to gamble."
"He went to the casino gambling."
The second sentence could mean he gambled on the way to the casino (which sounds kind of odd). You also need to be sure you're using the right preposition:
"He went dancing at Club 64."
"He went dancing to Club 64."
Again, the second sentence gives the impression he danced all the way to Club 64 (which sounds hilarious).
Student or Learner