one of the only men?

[Joe]

I saw a sentence like this:
I wore a tie. I think I was one of the only men there wearing a tie.

Does "the only men" work? I assume that there is only one man. Thanks.

informally it's is fine, and no, there is definitely more than one man

technically, well it's an interesting question i never thought of