Not a teacher.
I take note of the other opinions here, but I'll stand by what I've posted. In my experience the not uncommon form, "I wouldn't like to meet him in a dark alley", usually follows something like "Just look at that face!", or "He's a bit scary looking". That is, it's to do with the person's appearance. Similar phrases, more like the OP's example, would be "What an ugly bastard!" or "He's an ugly bugger". So, "He's an ugly bruiser. I wouldn't like to meet him in a dark alley", seems to me to be of this nature, a comment on appearances. Perhaps we'll just have to agree to differ.