That may be true in some states, but in California, the poll workers do not do the actual counting. Poll workers do count the actual number of voters who voted and then deliver the sealed votes to the Registrar of Voters (county level). The office of the Registrar of Voters hires people who either feed the ballots into a machine which does the counting, or, if the ballot is damaged in any way (folded, torn, ripped), they will count the votes by looking at the ballot. There seems to be no standard way that votes are counted in the US. Each state has its own rules and regulations and the responsibility for conducting an election is usually given to a county ( a political sub-division of a state).
We always had mechanical voting machines so technically the machines did the counting. The local poll workers just report the tallies from the machines to the county. Now it's electronic machines, so who knows if they are still counted locally or if they all get networked together for counting.