Especially since you aren't addressing the judge in court, you can safely address him simply as "Judge" or you can use his last name and say, "Judge Smith," for example.
"Your honor" is mostly used in court, unless you want to be very, very formal.
Please note that I am answering this question for judges in the U.S. I don't know what the customs are in other countries.
Debra (former lawyer, now an ESL tutor)
Teaching ESL to Adults
- For Teachers