that girls would participate actively only in a class that were all-girls
(Is 'were' a typo - it should be 'was'.)
The position of 'only' in a sentence can change the meaning, and render the sentence ambiguous. Take these:
'I saw her only once' : 'only' stresses the single instance that I saw her.
'I only saw her once' leaves it unclear whether I would often hear her talking next door (but never actually 'saw' her in person except one time; or talked to her often on the phone, but only met her (= saw her) once. This sentence is equivalent to saying, "I only actually saw her the once for a date (but we corresponded a lot)."
I saw her only once: it emphasizes 'once' - not twice, half a dozen times - ONCE.
In the sentence you chose, the position of 'only' means that it emphasizes 'in a class'. It is not whether the education would be conducted in some other format, such as small tutorial groups, that roused the parents' fears and objection so that they were insisting, 'we must only have classes, none of these small tutorial groups, because girls won't participate unless they are actually in a class/classroom situation'. No- they were afraid that if boys were admitted to the school, the girls would be reluctant to participate when boys are also present in classes. Hence:
..they would only participate actively (in class, in the lesson) if it was just all girls.