I think that it isconfusing since there is a bank and I have one brother, it is strange to start saying "No".
Thank you all
It is certainly wrong to answer "No" in the first case which uses "any".
One example is enough for "any".
If the question were "Are there some banks in this street?", you might answer "There is only one", but you still wouldn't say "No"; and people would think you were being perverse if you said "No, there is only one".
The same applies for the "brothers" example. You would risk being called sarcastic if you said "No", although it's alright to say "Only one".
I agree that the negative doesn't really fit. The person is not asking whether there are two or more banks- just one will do if that is what they are looking for, so answering 'no' is a bit confusing. If I were answering the question, I'd also remove 'only', because if there is one bank, there's no need for the negative connotation unless the person asking is angry or frustrated. If it's a normal question for help, I'd say 'Yes, there's one up the road on the left' or something.