It's a good question. 'There' might be preferable to 'it' in your New York example, but I agree both sound doubtful in some way, especially since you change from the present perfect to the preterite. Quite common in AmE, but not such eloquent speech. In any case, the first sentence of your example actually reminds me of artificial text-book speech, the type of utterance you're not really likely to hear in any case, but which may appear in a learner's phrase book.
Formal: "I have been to New York. In fact, I've visited [there/it] three times."
Informal AmE: "I've been to New York. Three times."
Interested in Language