'Family' can be followed by a singular or plural verb. A singular verb is used if you see it as a whole unit, and a plural verb if you see it as a collection on individuals. The same applies to a number of other words.
Verona is right.
However, I think that in your particular utterance, 'the Browns' would make it more likely that the following verb is 'live'. The speaker is implying that he sees the family as a collection of individuals.