The final part of the sentence, "unless you are trying to match", refers to your opinion about matching blue jeans and blue shoes.
He/she suggests that you wear grey shoes unless (except if) you want your (blue) jeans to match your (blue) shoes.
You can't get a job unless you've got experience (= you can only get a job if you have experience).
Unless you call me to say you're not coming, I'll see you at the theatre (= I will see you there if you do not call to say you are not coming).
(Definition of unless conjunction from the Cambridge Advanced Learner's Dictionary & Thesaurus
© Cambridge University Press)