Both idioms are related. The place where the sun never shines is your backside, and that is where you might tell people to "shove" or "stick" something that you do not want.
Often we shorten idioms, especially rude ones: "Take this job and shove it [where the sun never shines]".
It's a very forceful (and rude) way to refuse something. For example, you might tell your boss to "take this job and shove it", but only if you were really angry, and if you were absolutely certain that you will never work for him or her again.
Student or Learner