This story
happened with / is about a guy who

once

was sleeping (
Better as just 'who fell asleep' - you've said it is a story, so it must have happened [or be said to have happened] 'once'.) during a lecture. He finally woke up at the end of the class,
finding out that there were 'and saw/noticed...' (he didn't 'find out' - discover - anything) three mathematical problems written on the board. He wrote them down in order to do them as homework.
He beg
uAn solving the problems and to his surprise they were
so 'very/extremely/incredibly...' (or 'so difficult that it took him hours (or whatever) 'to solve them'; this sort of 'so' needs to be followed by 'that...') difficult to fathom. Solving them took him a very long time but he finally
could solve them / found the solutions to them.
(Note that 'found' is a correction of 'could solve' and not just 'solve'.)
The next day he went to his teacher complaining that these three exercises were so hard,
Note: this comma is important, it signals the fact that this is not an incomplete 'so ...that' expression. but he had
been able to solve them / found the answers. The teacher asked him: "Which problems are you talking about?"
"The ones you wrote on the board ", the student answered.
The teacher told him that he had not wanted them / the students to solve these cases
(Maybe maths students talk about individual problems as 'cases', but the word choice seems a bit odd to me.) The fact was, these were
those (This correction was right. Saying 'those' makes the selection exclusive problems: as in 'Those students who have forgotten their dinner-money, see me after the class.' If you say 'those problems that no one..' then there are only three unsolved problems in the world!) that / these problems were
those which 'ones that' no one around the world had been able to solve!!"
This story tells us that everything we do depends on our thinking and on our belief
of 'in' or 'about' our abilities. If he had not fallen asleep while the teacher had been saying that these were

the most complicated problems
'unsolved', he would not even have thought of trying to solve them. "