Could the reason be because it is always meant to be a pair ?
Does your example extend to the others (jeans, pants and trousers)?
Can shorts also mean underwear?
Yes, the full phrase for all of those is "a pair of........" although you don't have to use the whole phrase.
I have a new pair of trousers.
I have new trousers.
He's wearing a pair of green trousers.
He's wearing green trousers.
I'm going to buy a new pair of jeans.
I've just bought new jeans.
In the US, I think shorts can be used to mean underwear. Generally, in the UK, you would have to be more specific - boxer shorts - to differentiate from the type of shorts you simply wear in the summer because they're cooler!
There is also a difference in meaning between the USA and the UK with pants. Trousers in the UK are pants in the USA. Pants in the UK are underwear (men's and women's, althoug women's are regularly called "knickers"). I'm not quite sure what the AmE word for women's underwear is (possibly panties?)