Look at this conversation:
She: What are you doing this afternoon?
He: I have to go to band practice.
Here, 'have to go' means that he has a regular time to attend at band practice, but there is no urgency/big deal about it. Its just a commitment later that day.
She: There's a good movie on TV. Can't you skip it?
I have/I've got to go. I missed last week and the band leader will be furious if I don't show.
Now, 'got' gives the statement a sense of 'must'
She: Are you sure? It's such a good movie.
He: No, I really must go.
This now stresses the absolute importance of going.
Look at this conversation:
Mother: Come back and finish your breakfast!
He: Sorry Mum. Gotta go.
"Sorry' here is short for 'sorry Mum but I can't do what you are asking'
...and as Tdol said, 'gotta go' is colloquially short for 'have to go' - there is something more important than finishing breakfast he has to rush off to ( the same way band practice was more important than seeing the movie)