Can we drop "more" in the usage of "couldn't agree with * more" to show we "agree"?

As many people know that when we say "I couldn't agree with you more" means "I absolutely agree with you.", one of my friends used it like this:

A: The coffee is quite good, isn't it?
B: Yes, I couldn't agree with you.

I know at that time he wanted to say "I couldn't agree with you more." but without "more" it sounds a disagreement in the past and all the examples I know in this kind of usages "couldn't" will always put with a comparative, such as "Couldn't be better", "Couldn't be worse"....

Can the "more" be droped out?

No; it doesn't make sense to me without 'more'. ;-)

tdol,

"I couldn't agree with you more" means "I agree with you wholeheartedly"
"I couldn't agree with you less" means " I don't agree with you at all", right?

What about, "I couldn't agree with you any more"? What does it mean?
Also, what does, "I couldn't agree with you any less" mean?