If you say you are terrible with names, you mean to say that you aren't very good at remembering people's names.
Here's the definition of quirk:
1. A peculiarity of behavior; an idiosyncrasy: “Every man had his own quirks and twists” (Harriet Beecher Stowe).
2. An unpredictable or unaccountable act or event; a vagary: a quirk of fate.
3. A sudden sharp turn or twist.
4. An equivocation; a quibble.
5. Architecture. A lengthwise groove on a molding between the convex upper part and the soffit.