Some Japanese English teachers say (a) is wrong and (b) is correct, but I do not understand why (a) is wrong.
(a) I'm very exhausted. (b) I'm quite exhausted.
a little exhausted' being used. To me, it's one of the adjectives in the grey area between gradable and non-gradable.