If I were to say "he swang the bat" or "he swung the bat" both would be correct right?
So what is the difference between past tense and past participle?
There is no such word as "swang", so only "He swung the bat" is correct.
The past tense is verb tense expressing action or state in or as if in the past: a: a verb tense expressive of elapsed time (as wrote in “on arriving I wrote a letter”) b: a verb tense expressing action or state in progress or continuance or habitually done or customarily occurring at a past time (as was writing in “I was writing while he dictated” or loved in “their sons loved fishing”)
The past participle is a participle that typically expresses completed action, that is traditionally one of the principal parts of the verb, and that is traditionally used in English in the formation of perfect tenses in the active voice and of all tenses in the passive voice
[cf MW]