We have Mothers' Day to honor mothers, and Fathers' Day to honor fathers, and Veterans' Day to honor veterans. Yet with Labor Day, which honors those who labor, we have no apostrophe.
I'm not a teacher, but I write for a living. Please don't ask me about 2nd conditionals, but I'm a safe bet for what reads well in (American) English.
I know people don't find it persuasive, but that's what "that's the idiomatic way to say it" comes down to. There is not a lot of logic in the way we handle these things. Generally, when one noun modifies another, we use the plain noun. Yet it's Veterans' Day. (Even less logically, many official sites have it has "Veteran's Day, as if there were only one.)