(Not a Teacher)
The exact meaning of "at the best of times" is not entirely clear. Personally, I think you're suggestion is on the money, but let's suffice it to say that it's just emphasizing the fact that he is hard to understand.
"By far" means that he far outstrips the competition when it comes to the obscurity of his writing.
Student or Learner