a) a man who is handsome looks attractive [= good-looking]: an extremely handsome young man
Sam was tall, dark and handsome.
his handsome face
b) a woman who is handsome looks attractive in a strong healthy way
smart can mean intelligent (men + women), and also neat and fashionable (men +women), e.g.
You're looking very smart = you're wearing neat and attractive clothes and have a tidy appearance.
(from: Longman Dictionary of Contemporary English)
Today few people will insist it depends on the gender of the person any more with the word smart. Beautiful and handsome are gender specialized, however, to most people. Although you will see "handsome woman" in Jane Austen at times, and you can hear "what a beautiful man" but this would mean a kind man.