This could be one of those things that varies quite a lot, both regionally and person to person, so I can only speak for myself.
Actually, I probably wouldn't use any of the (correct) terms you mentioned. I'd be most likely to call them an 'alum'. "He's an alum." "She's an alum." I might also use the example you gave- "We went to the same school." or "We were in school together." (if we were at the same school at the same time).
Of the four words you mentioned, alumni is used far more than any of the others- even in reference to a single person, male or female.
Student or Learner