Last year, Democratic Rep. Anthony Brindisi lost his House seat by a razor-thin margin of just over 100 votes. While the outcome was disappointing, even more ballots were rejected because people voted at the wrong polling site. 

"What was more frustrating to me was knowing there were hundreds of legally registered voters who cast their ballot in my election, but through no fault of their own, never had their vote counted," Brindisi said on Wednesday. 

It's known as wrong church, wrong pew voting — voters turning out at the wrong polling site, filling out a provisional ballot because they don't appear in the voter rolls, only to have their ballot rejected. Brindisi is backing a change to New York's election laws that would allow these votes to be counted and avoid disqualification. 

A bill advanced this week through the Senate Elections Committee to address the issue.

"Rather than being redirected as the law requires, a poll worker is directing them to complete an affadavit ballot," said Jarret Berg of the group VoteEarlyNY. "Frankly, we think both people in that transaction believe the ballot will count and the voter might be in the right place. But after the fact, that ballot is disqualified in full."

A recent review by Berg's group found 13,800 votes were rejected because they voted in the wrong location. All of their votes were discarded, even if they were still able to vote in a congressional race or for president. 

"We knew it was a substantial amount, but this really blew away whatever our expectations were," he said. 

The lion's share of these rejected ballots are in New York City, disproportionate to the rest of the state. Still, in narrow races like the 22nd Congressional District last year, dozens of rejected ballots can make all the difference. 

Berg called this is a statewide problem that can be addressed with legislation currently under consideration in the state Senate and ensure these votes are at least counted. 

"We see it in Erie, in Monroe, in Onondaga, in Albany, in Dutchess, in Ulster," he said. "Places that have population centers — these are the counties outside the city that are he hardest affected."