If your wife is the mother of your son and daughter then you could say I live with my wife and our son and daughter. Or just as you have said. If you put my in front of son and daughter then it implies that your wife is not their mother. But I think I may be over pedantic
