'Passed' is a verb - the past form of pass - move past something, in a direction or to the other side of something, so you can pass something on your way.
So, "the humor passed me by" is correct, in this sentence 'pass' is the verb.
'Past' can be an adjective and a noun, as opposed to present. (Consider phrases like "in past ages", "past events" etc.)
'Past' is also used as a preposition or an adverb, and this is what you need. It means something like after, further, to the other side of something. It gives information about a direction.
This way, "time rushed past" is correct. "Rushed passed" would be illogical, since you'd have two verbs. However, you could say "rushed, passed", which is "rushed and passed".
Student or Learner