"A large crowd used to collect / gather to listen to his speech."

Which word should I use?

Thanks.