Student or Learner
Do you think it is wrong to say "pay by cash"?
I checked COCA and the number of hits are 3, while "pay in cash" has 48 matches.
If a second language learner asked you if it would be fine to say "pay by cash", would you say "Yes"? Or would you advise them to use "in cash"?
I agree with COCA. (HOW COULD I NOT?) "Pay in cash" is by far the more common but "pay by cash" is heard occasionally and understood by fluent speakers of AmE. I would advise learners to embrace mainstream usages and avoid the less common. Sometimes, although not in this case, there may be subtle differences in meaning.