Student or Learner
Iīd better not catch YOU doing that again!
The children donīt enjoy YOUR teasing.
I canīt understand YOU/YOUR making such a fuss.
Is there a rule for the use of either the accusative or the possessive between the verb and the -ing form?
It's a bit of a grey area. In the first of your examples, 'you' srikes me as the only possibility because it is the object of the verb catch.
Traditional grammars argue for the possessive with gerunds, which is logical, but it is so widely ignored by native speakers that it is almost impossible to argue a case for it being a rule IMO, which is why you'll see people say that it is preferred in formal style.