First of all, I'd choose a particular phrase and be clear on how it's used, as there are many ways of expressing regret.
The verbcan be used in more than one way:regret
Regret + infinitive- where the speaker is regretful before the action:
I rgeret to say that your application has been unsuccessful.
Regret + gerund- where the speaker is regretful after the action:
I regret being so rude to her in the meeting yesterday.