Usually, I would say simply "He is riding a horse."
I wouldn't say that "He rides on a horse" is incorrect, but that it's not natural.
Just like you might say "I saw him driving a convertible" instead of "I saw him driving in a convertible." It's not wrong, but it's just not often said.
[business writer, not teacher]
Student or Learner