In the U.S., the court will allow however much incriminating evidence that is available. For example, if they have a surveillance tape that clearly shows a man shooting another, in addition to that damning evidence the prosecutor would still include fingerprint evidence and anything else he has to strengthen his case. The goal is to present an "airtight case," meaning there is absolutely no room for doubt.
However, I'm wondering if your "one bug in the loaf, discard the whole loaf" idiom is referring to the "beyond a reasonable doubt" rule in U.S. law. That is, even if there is a lot of very convincing evidence, if there is one small piece of the puzzle that doesn't quite fit - one detail that leaves a question of guilt in the jury's mind -then the defendant will be found "not guilty." The jury is always admonished to make their decision only if they are 100% convinced of guilt, or "beyond a reasonable doubt."