In reference to. It refers to some other document that has already been sent. While the subject is the overall reason, the reference is to a specific document.
Subject: Allegation of underpayment of taxes
Ref: Your letter of Dec 22, 2010, file #1233445
Dear Mr. Bowman:
In my experience, the "Dear Sir" or "Dear Mr. Smith" comes below both the subject and the reference line.
Student or Learner