"Bicep" is the singular noun. You have one bicep in each arm so, in total, you have two biceps.
Is "the biceps of his right arm" incorrect but is "the bicep of his right arm" correct?
Nope. You have two biceps: one left biceps and one right biceps.