You have good ears! It is possible the native speaker said "I forgo' my mom's birthday", perhaps using a glottal stop instead of a t, or even missing the t completely.
Yes, especially when speaking quickly, native speakers will merge consonants together...
last night loses the t
I haven't been to Wales loses the t
But in a formal situations the t would be pronounced, especially if the speaker was speaking slowly.
Your rule that says you must pronounce the letter at the end of all words is only really true for words ending in consonants, e.g. pig, run, sound, etc.
Many English words end in a vowel which isn't pronounced, e.g. website, everyone, have, etc. etc.