***Neither a teacher nor a native speaker.***
Both sentences have a completely different meaning!
He used to get angry with his brother when they were little.
Once (when they were little) he used to get angry.
Now (they are not little anymore) he doesn't get angry anymore.
He would get angry with his brother if they were little.
Now they have no problems with each other.
However, imagine they were kids.
Then he would get angry.
As said, grammatically seen both sentences are possible, but their meanings are nothing alike.