The first option is incorrect because he might have come to the office more than 3 times (and I wasn't in the office so I didn't meet him).
The second option might be incorrect because I might have met him more than three times, but not in the office. (For example: "I met him four times, three of which were in the office")
Therefore, option 3 is correct - Neither.
Student or Learner