In most important areas of life, wisdom or "knowledge" is best. The idiom you found is often used for circumstances in which knowledge is simply not necessary because the matter is not serious. Here is an example. If your friend gave you a nice gift, you might wonder how much did he pay for it. Do you really need to know? Is that knowledge important? No. But you just can't help yourself and you sneak a look at the receipt. HEY HE ONLY PAID $5 FOR THAT! And you thought it was worth much more. You might feel less grateful. In that situation I would say, "Where ignorance is bliss, it is folly to be wise." You were happy not knowing, but foolishly you found out.
