What heidita says is right; there are two cases, and each form belongs to one case: generally, plural - closest friends; singular - the closest of [someone's] friends. However, there
is a usage - perhaps slightly archaic, or at least rather arch/artificial
(
one of those A- words!

) that puts 'the closest of' with the plural - "We used to be sworn enemies, but now we are the closest of friends"; the "of" form goes with the singular in this case, because the utterance is dealing with a single (albeit two-way) relationship.
Also, the "of" form can go with the plural in cases like this: "If you want to keep a secret, don't share it with even the closest of your friends; any one of them may blab."
b