If it is asked to a group of people, then the answer would be in plural and it would be "Yes, we are."(The group may be a family or a group of tourists etc.)
Both are not correct but depending on the situation to whom the question is asked would the answer would be correct.
P.S. : I am not a teacher of English, but just someone who loves the language.