That's a good question. A "cake shop" sounds pretty natural to me. Certainly I would understand it (I'm a native English speaker from the United States).
However in all my life I have only seen or heard about one store that sells nothing but cakes, and that's in my native Philadelphia, where this guy sells carrot cakes of all sizes...and everyone in the neighbourhood says "the carrot cake man". E.g.: "Let's go get something for dessert at the Carrot Cake Man".
People can be so unpredictable, can't they?
auhtor of The Secret Lives of English Verbs