That is one of its meanings, yes, reffering to a kind of pudding that looks (slightly!) like a buttock.
It is also a bad swing in golf, and by association anything, including people, that doesn't work very well.
"He's a duffer. he never gets anything right."
"My MP3 is duff. I need a new one."