I must have been watching the game is a good equivalent.
I suppose you can make a different set of circumstances.
If you hadn't asked me to go out with you tonight, I would have had to have been at home, entertaining my annoying cousins. In this case, I am out with you, so I am not at home, and my obligation to entertain them does not exist (had to have been = obligation )
In the original, I read "had to" as "it must be the case, because nothing else makes sense" as in "Look at this vase - it has to be at least 200 years old."
So, 'would have' could mean you did it, or you didn't, couldn't it? Is it context-dependent? It seems (to me) as though if you feel there is an unwritten if-clause behind, the sentence oftentimes carries the sense of 'you didn't,' and if you don't, the sense of 'you probably did,' doesn't it?