Native speakers might say that. It's basically a metaphor, as you can't literally drop the weight (allowing it to fall as if from your pocket). If your accent is poor, and doesn't inspire confidence, people will be unsure what you mean. If your accent is great, they will be confident they have understood. Does that help?
Student or Learner