I'm okay with either 'fish shop' or 'fish store' in AmE, but I'd expect them to be a place that sells live aquarium fish, not fish for human consumption.
If I'm buying fish for dinner, then I'd be looking for a seafood market. We don't have those in my part of the US unfortunately, so the closest thing I have is the seafood section (I think they call it the seafood department) in one of my three local grocery stores. The other two grocery stores only have seafood available in the frozen goods section.
Even the stuff in the seafood section is frozen product that's just thawed out for display on ice. None of it is fresh in the sense of "never been frozen". The cooked cocktail shrimp even occasionally still has some ice crystals in it. Shrimp just shouldn't crunch unless it's fried...
We're just too far from any major water sources for actual fresh-caught seafood markets. It all has to be frozen to survive the shipping distances to my region, sadly.