Can you please say if the tense here is OK?
Did you go to abroad last month?
Yes it was the secong time for me to go to Japan with Jim.
 Did you go to abroad last month?
The tense is OK, but there's no need to add "to". The word "abroad" already has a directional prepositional housed within. It's "a-".
Did you go abroad last month?
Yes. It was the second time I went to Japan (with Jim).
Note, "with Jim" is additional information, so it should be added as such, like this,
Yes(, I did). It was my second visit, with my friend Jim.
Note, words in brackets (. . . ) are optional.
All the best.