***Neither a teacher nor a native speaker.***
Let's create another example.
You are an actor and I have to mark/rate you.
You are really good - this is a fact.
In this case, I can only say:
If you weren't so good, my decision would be easier.
I think it's not meaningful to say, "If you aren't so good, my decision will be easier." because I'm already aware of your skills.
Also it makes no sense to say, "If you hadn't been so good, my decision would have been easier." because it's present - we are talking about now.
Your example is present as well, because he quotes himself.
(While he was watching you he was thinking something.)
One week later you ask me, "Nightmare, what do you think about my skills?"
I say, "You are great, and I told myself, "If you weren't so good, my decision would be easier."
I hope I could answer your question...
If it's still unclear, just ask and I will think about it again
Student or Learner