Here is the phrasing.
It's hard to to X, much less Y = It's harder to do Y than to do X.
It's hard to do something to X, much less Y = It's harder to something to Y than it is to do the same thing to X.
He is hard to persuade. His parents are harder to persuade, even though he is hard himself. .
It's hard enough to get him to exercise once a week, much less once a day. It will be harder to get him to exercise daily than weekly.
It's hard for me to talk in public, much less sing. It's much harder to sing in public than to talk in public, which is also hard.
Student or Learner