There are some small differences, plus the gerund and infinitive are not always 100% interchangeable in all cases. Verb tense governs usage somewhat.
With 'begin', you can use the gerund or infinitive in non-continuous tenses, with little to no difference in meaning.
I began to cry when I heard the bad news. (crying started and continued for unspecified time)
I began crying when I heard the bad news. (crying started and continued for unspecified time)
With 'begin' and continuous tenses, you can only use the infinitive.
The baby is beginning to cry. (crying has commenced and is on-going)
Similarly, 'start' can use both the gerund or infinitive in non-continuous tenses, although there's a possibility of difference in meaning.
With 'start' and a non-continuous tense, there's simply an indication that the action in the gerund has initiated.
Tom started crying when he heard the news. (crying started, with no indication of duration)
In a non-continuous tense 'start+ infinitive' usually suggests that the action in the infinitive wasn't completed or continued.
I started to cry but quit when John walked in the room. (crying started, but was prematurely halted)
However, with 'start' and a continuous tense, you can again only use the infinitive.
Roger is starting to cry about his lost puppy. (crying has commenced and is on-going. )