When you come to a store and you want to pay with a credit card. What happens to it? They "wtihdraw money from your card" or "get money off your card".
You pay 'with plastic' - but that's the only case I can think of where the physical thing is involved in the process.
Note that there are cards that you 'load'. In that case the physical thing is involved. At some outlets you can 'put money onto your card' - although more formally, again, that would be referred to as 'making a credit to your account'.