"Sheepish" originally meant simply "resembling a sheep in some way". By the end of the 17th century, it had acquired its present primary meaning, "bashful and diffident and lacking presence of mind among other people".
Presumably the reference is to the character of (female) sheep. They tend not to look you boldly in the eye or stare at you arrogantly or block your path across their field in a swaggering sneering kind of way. Instead, they scuttle off in ignominious panic.