If you mean a woman who sleeps with a man who is not married to her -- a mistress, in other words -- the male equivalent might be "lover".
But if you mean a secondary wife or a low-ranking member of a harem, I don't think there is a male equivalent. As far as I know, societies where one woman can (legally) have more than one husband are very rare. I would probably write "lower-ranking husband" or "additional husband" if necessary, otherwise just "husband".
'Concubine' is derived from 'concupiscence' which means sexual desire.
Both terms are very old but they're still used in some languages derived from latin. (concubine, concubinage in French / concubina ,concubinaggio in Italian) however I do not think I've heard the term in spoken English before.
I'd use lover (both male and female) and mistress (female) if it was an extramarital affair.
Verb to cohabit means that two people who are not married, cohabit, they live together and have a sexual relationship. It's a formal term anyway.
However, a cohabitant/ cohabitee is a formal term for someone who lives in the same house, apartment, etc. as someone else, it doesn't necessarily involve a sexual relationship.
Yes, it's strange when some words are obvious in a language like for example concubin in French which is the masculine of concubine (= concubine). And in English it does not seem to have a fully similar translation. Because I believe that the word lover, is masculine and feminine and gigolo is a quite pejorative word while concubine is not.