Close only counts in horseshoes and grenades/Atom bombs
It's used sarcastically to say that an answer or a result that is close to the expected/desired/correct answer or result is not good enough.
If I tell you a phone number is 610-555-8727, and it's really 610-555-8729, the number I gave you is "close" to correct, but it's useless to you. If I said "Oh, sorry, but I was close" you might say that expression back to me.
In the game of horse shoes, the shoe closest to the pole gets points, so being close does count. If you drop a huge bomb close to your target, the target will get hit, so close does do the job.