The both applies to the two nouns, "boys" and "girls". It wouldn't matter whether or not the nouns were singular, the both merely emphasises there's two of them.
So "Both the men and the women were singing." is fine, but "Both the men, women and children were singing." is not.