There are many problems with travelling through time. The commonest are the so-called paradoxes.
Yeah, like what happens if you meet your grandfather? (It doesn't matter which side of the family.) What if you should happen to kill him before he met your grandmother? That would mean that you had never been born. That would mean, of course, that you couldn't have traveled back in time and you couldn't have killed your grandfather. Then, of course, you were born after all.