It means "You're right. I agree that we should not do that." (That=leave without telling anyone.) Often it comes with a feeling of regret.
We can't eat the entire cake Mom made for dinner tonight.
No, I suppose not. (No, you're right. Eating the entire cake is a bad idea.)
You can't just waltz in there and say you want a raise!
No, I suppose not. (You're right. Doing that is a bad idea.)