In most cases, it doesn't matter which one is used.
I started swimming.
I started to swim.
He started singing.
He started to sing.
They started buying groceries at a different supermarket.
They started to buy groceries at a different supermarket.
However, as always, context is everything. If those sentences above were followed by a period of time, the meaning might change. For example:
I started swimming when I was seven years old = I learnt how to swim when I was seven years old.
I started to swim when I was seven years old = I have been swimming constantly since I was seven years old.
He started singing last year = He decided to take up singing as a hobby or to have lessons last year.
He started to sing last year = He has been singing constantly since last year (and would have a very sore throat by now!)
Can you see the difference between the following?
I started playing tennis ten years ago.
I started to play tennis 15 minutes ago.
Student or Learner