Hello again,
The use of "whom" is very traditional, and is not as commonly used as it used to be. Traditional grammar also says that you must never end a sentence with a preposition. So, if you insist on using "whom", you would then be expected to not end your sentence with the preposition "to". This would be done to maintain the formality of the whole sentence, rather than to mix formal with less formal.
That is why, when my grandmother was speaking on the phone to someone she didn't know, she would have used the very formal and extremely polite "To whom am I speaking?", rather than "Who am I speaking to?" [informal], which in those days would have probably appeared rather abrupt, if not rude.
The argument about whether, or not, sentences should be ended with prepositions has rumbled on for years. There is a [probably untrue] story about Churchill that illustrates this quite well:
After an overzealous editor attempted to rearrange one of Winston Churchill's sentences to avoid ending it in a preposition, the Prime Minister scribbled a single sentence in reply: "This is the sort of bloody nonsense up with which I will not put."
http://itre.cis.upenn.edu/~myl/languagelog/archives/001702.html
The point he [or the person attributing the words to him] was making was that, sometimes by applying grammar rules to the letter you may end up with sentences which both look, and sound, unnatural. That is why there have always been those in favour of writing in a more natural way, by ending sentences with prepositions, and those who prefer the more traditional method of writing.