Generally, you're right and 'Yes, I'm not bad' would be a very strange answer, but occasionally when people just say yes or no , you will hear answers like Butch's. It's like singular and plural, which theoretically should be very simple, but in practice can be more complex. I think the complexity stems from the fact that not only do we have a negative statement + positive tag question, but there's also an adjective with negative connotations. If you asked someone 'Aren't you good?' the answer would be easier in some way than if you asked them 'Aren't you bad?', where there might be a pause while they worked their way through it.