Student or Learner
Can I ask in a supermarket, 'What aisle is milk'?
How else would you say that?
In BrE, "Which aisle is the milk in?"
Answer: "It's in aisle six".
Remember - if you don't use correct capitalisation, punctuation and spacing, anything you write will be incorrect.
But, as an ellipted version of 'What aisle is [the one with] milk [in it]?' or 'What aisle is [the one where] milk [is displayed]' I'm pretty sure I've heard the OP's version. It seems to me to be talking about the layout of the store, rather than the whereabouts of the aisle with milk in it. (Maybe, though, the speakers I remember weren't native speakers...)
What I ask would be 'Where would I find milk?' or 'Could you point out the milk aisle?'
We'd say "in" where I live too.
I'm not a teacher, but I write for a living. Please don't ask me about 2nd conditionals, but I'm a safe bet for what reads well in (American) English.