You ask me
"What would you say about these?
We walked a mile.
"Mile" is marked as a noun by the indefinite article that precedes it.
We walked fifty feet.
"Feet" is marked as a noun because it is modified by an adjective.
This is my answer
We consider 'Miles' as a noun. But sometimes we need to use it as an adverb like that I mentioned.
'four miles' or 'fifty feet' is one cluster fuctioning as an adverb. It means 'miles' and 'feet' are a noun, it is modified by an adjective, and the cluster functions as an adverb.
If we frequently use 'miles' as an adverb, a dictionary hold the function of an adverb of 'Mile'. and then It can be modifed by an adverb. But It is not usual, and there is only a noun in a dictionary. For example, If people often use 'Millennial' as a noun, it become usual. It can be a noun. That's why there are many function of one word in a dictionary. I believe it is one of the mechanism of a language.