There is no difference between "pack" and "pack up" in that use. Sometimes we use more words than are strictly necessary. It's patterns of language use. For example "get off of my car!"
I don't know what you mean by "down set." Song lyrics are not a good choice for studying language. "Off" and "down" are not supposed to be grouped together. Consider this rearrangement: "Off he went down Brewer Street."
Student or Learner