1) London is the place to be.
2) London is the place to be in.
3) London is the city to be.
4) London is the city to be in.
Of the first pair, #1 is by far the more natural. (#2, although theoretically grammatical, is only marginally acceptable.)
Of the second pair, only #4 is fully acceptable. Again, despite theoretical possibility, most speakers would reject #3.
As to the reason...
It revolves essentially around the kind of noun that can stand as the antecedent of the relative adverb 'that', to determine which we have to reconstruct canonical (putative original) forms such as the following:
 This is the place (that) I was born.
This is an acceptable, albeit informally constructed, sentence, where 'that' - more typically ellipted - functions as a relative adverb of place (i.e. meaning 'in/at which'), whose only normally possible antecedent is the noun 'place', so that, in contrast,
 *This is the city (that) I was born.
is not possible.
And these same factors tend to influence the acceptability of adnominal infinitive phrases such as those of your original sentences: the acceptability of  accounts for that of #1, while the unacceptability of  accounts for that of #3.
A couple of additional observations:
1. #3 is theoretically possible as a sentence, but only as one in which 'place' stands directly in a subject-complement relation to 'London', i.e. where the overall meaning is, not sensible
London is the place in which one should be.
!London is the place which one should be.
The concept of recommending that someone should/could actually become a city - let alone a particular one - runs so inherently contrary to sense that most speakers would not consider the sentence usable under any set of circumstances, hence, to all practical intents and purposes, unacceptable.
2. The strangeness/infelicitousness of #2 is not easy to account for, as, being structurally analogous to a wide range of possible, natural sentences, e.g.
 A spoon is a thing to eat with.
 Bears are animals to be wary of.
 This is a problem to think about.
, it certainly does not offend against any general rule of syntax.
However, in  - , we note that the final preposition is obligatory to avoid the production of a sentence that is either ungrammatical or semantically absurd, while, as already established, quite the reverse applies to #2. The idiomatic tendency in such cases would thus seem to be: omit a deferred preposition where it is possible to do so without prejudice to either structure or sense.
Student or Learner