Are both the following sentences correct?
He failed math.
He failed in math.
Example  answers the question, what subject?, whereas example  answers the question, where or what subject area?
 He failed math.
What subject did he fail?
 He failed in [the subject area] math.
Where [in his courses] did he fail?