"We'd" can be "we would" or "we had." You need to know the grammar to know which one is meant.
We'd better = We had better. The sentence "We would better go" is not grammatical, but "We had better go" is, so you know the 'd is "had" not "would."
(I would be very unlikley to actually tag that statement with "hadn't we" -- it sounds very unnatural to me. But it's grammatically correct.)
An example of "We'd" being "We would":
We'd know one way or the other, wouldn't we? = We would know... woudn't we.
Student or Learner