In the phrase "the 20-minute trip", "20-minute" is used as an adjective. As you know, adjectives are not plural in English, so "the
20-minutestrip" would be incorrect. (As you'll have noticed, I used a hyphen to connect 20 and minute. This is not necessary, but it's very common. I also find that it helps to remind one that this is an adjective and therefore the plural form of the noun is wrong.)
In the second example, "20" functions as the adjective and "minutes" is a noun. Because 20 is clearly plural, "minutes" is correct.
I hope this helps.