We use plural nouns when we talk about plural things. And we conjugate verbs according to the normal rules.
Someone can certainly be caught "taking a bribe," but usually someone who engages in that kind of behavior does it more than once. And police would try to implicate a person more than once, to build a stronger case. So taking "bribes" is more normal.
Interested in Language