Skip to Main Content

2016 General Election: Post-Election Tabulation Audit Overview

Introduction

Following the 2016 General Election, this office will conduct a post-election tabulation audit.  The tabulation audit will be conducted in addition to the comprehensive audit that is conducted following each election.  The purpose of the post-election tabulation audit is to verify and confirm the accuracy of the primary voting system’s reported results.  A post-election tabulation audit is not a canvass or a recount; it is used to verify that the primary voting system accurately tallied ballots and that the winners of each contest were called correctly.

In June 2016, this office conducted a pilot program in Carroll and Montgomery Counties where three different post-election tabulation audit methodologies were tested1 A comprehensive report from the pilot program is available on our press room page under “Recent Reports”.  At the conclusion of the pilot program, an independent, automated tabulation audit was recommended for statewide implementation following the 2016 General Election as the most comprehensive, efficient and user-friendly of the three piloted methods.

An independent, automated audit relies solely on the use of independent software to tabulate ballot images. The results from the independent tabulation are then compared to the tabulation results from the primary voting system. Any variances between the two tabulations can be identified and resolved using the ballot images.  For the 2016 General Election, this office contracted with Clear Ballot, a Boston-based elections technology company which offers the ClearAudit software product. ClearAudit is the only currently available, market-ready software product that can perform an independent automated post-election tabulation audit using ballot images imported from another voting system.

The post-election tabulation audit will be conducted using ballot images, as was the pilot program.  The use of ballot images allows elections officials to maximize the technological functions of the new voting system while minimizing human error and eliminating chain of custody issues by using securely stored ballot images, rather than voted paper ballots.  The use of ballot images removes the need for election officials to physically handle or count voted ballots unless a petition for recount or other judicial challenge is granted.

Finally, a discrepancy variance level of 0.5% was established for the post-election tabulation audit2.  If there is in excess of a 0.5% variance in any given contest between the primary voting system results and the ClearAudit results, the adjudication of all ballot images from that contest by both systems will be compared.

Pre-Audit Tasks

To prepare for the post-election tabulation audit, SBE provided Clear Ballot with:

  • PDF files of all ballot styles for all 24 counties;
  • Pre-election reports for all precincts showing no votes cast;
  • Information to assign ballot styles to precinct; and
  • Files needed to process ballots used with the ExpressVote ballot marking device.

Upon receipt of the data, Clear Ballot:

  • Created a Ballot Definition Files for each county;
  • Validated each Ballot Definition File; and
  • Created a ClearAudit database for each county.

Because the post-election tabulation audit uses ballot images to verify the accuracy of the voting system, SBE staff members will verify – before the election – that the ballot images from the voting system are identical in content to the ballots cast by voters.  This verification will be conducted in addition to the required Logic and Accuracy (L&A) Testing conducted on all voting equipment prior to an election.  To perform this verification for the 2016 General Election audit, SBE staff will:

  • Generate a test deck for Baltimore City.  Each ballot in the test deck will have a unique ID number printed on the bottom of the ballot.
  • Scan test deck in a precinct scanner or high-speed scanner.
  • Print results from a precinct scanner or high-speed scanner.
  • Load results into the primary voting system’s Election Results Manager.
  • Acquire results and images in the primary voting system’s database.
  • Export all ballot images and Cast Vote Records.
  • Compare each test deck ballot against the ballot image and Cast Vote Record.
  • Record on spreadsheet unique ID number, Cast Vote Record number, and the result of the comparison.

Audit Process

On or before November 8, 2016, Clear Ballot will ship to each local board of elections an external hard drive.   The local boards of elections will save on the hard drives ballot images, in unencrypted form, from election day and early voting and return the external hard drives to Clear Ballot by November 10, 2016.

Once the external hard drives are received, Clear Ballot will:

  • Copy voted ballot images from each county into its corresponding audit database;
  • Perform an automatic tabulation of voted ballot images from election day and early voting;
  • Resolve unreadable ballots;
  • Perform an audit database review; and
  • Send a Preliminary Statement of Votes Cast to SBE.

Once SBE receives the Preliminary Statement of Votes Cast, SBE will no later than November 16, 2016, send precinct-level results from early voting and election day.   Clear Ballot will use this information to create the Comparison of Votes Cast, a report that compares results from the primary voting system and Clear Ballot.  This delay in sending the precinct-level results to Clear Ballot is intentional.   This procedure, in effect, results in a “blind” audit meaning that Clear Ballot will publish its tabulation before knowing the results from the primary voting system.

By November 18, 2016, Clear Ballot will provide each local board of elections with log-in access to its county audit database.  SBE will have access to all audit databases.

By November 21, 2016, each local board will save on a hard drive ballot images, in unencrypted form, of all absentee and provisional ballots cast and results from the absentee and provisional canvasses.  Clear Ballot will add these ballot images into the appropriate database, resolve unreadable ballots, and perform an automatic tabulation of voted absentee and provisional ballot images and generate a Comparison of Votes Cast for the absentee and provisional ballot canvasses.

After Clear Ballot provides the reports described below, the audit results will be available to the public and Clear Ballot will provide at least two public presentations of the post-election audit results (dates to be determined and posted on SBE’s website).    

Reports Produced by Clear Ballot

ClearAudit technology provides visual reporting tools that allow election officials to generate sortable contest, ballot and precinct reports that create a visual connection to each ballot image and provide detailed information about how it was adjudicated. Clear Ballot will produce the following audit reports for each county, which will all be available in each county’s audit database:

  • Comparison of Cards Cast for each canvass:  This report compares the number of ballots counted during early voting, on election day, during both absentee canvasses, and during the provisional canvass against the number of ballots tabulated by ClearAudit.  This ensures that ClearAudit tabulated the same number of ballots as the primary voting system.
  • Comparison of Ballots Cast by Precinct: This report compares the number of ballots cast in each precinct3 against the number of ballots tabulated by ClearAudit.  This is another way to ensure that ClearAudit tabulated the same number of ballots as the primary voting system. 
  • Comparison of Votes Cast: This report compares the results from the primary voting system against the results tabulated by ClearAudit and identifies possible discrepancies by candidate or choice. 
  • Contest Vote Discrepancy Threshold Report: This report shows - by contest - the number of vote differences between the two systems, the total votes cast by the primary voting system, and the vote difference as a percentage. Before the audit was performed, SBE determined that a percentage of 0.5% or higher would trigger an additional review, which could include a manual review of voted paper ballots.

Reports for each county are available here.

Conclusion

An independent, automated post-election tabulation audit method maximizes the use of technology in election administration, minimizes human error and handling of official election materials, including voted ballots, and provides the fastest, most comprehensive and most visual and transparent presentation of audit results.  The ClearAudit solution will be used to verify the accuracy of the voting system used for the 2016 General Election.  After the election, the effectiveness and accuracy of the audit on a statewide level will be reviewed and determine whether it will be used again in the 2018 election cycle.

If you have any comments on the 2016 General Election post-election tabulation audit, please submit them in writing by January 31, 2017 (the deadline was extended to receive more comments) to:

Maryland State Board of Elections
Attention: Nikki Charlson, Deputy Administrator
151 West Street, Suite 200
Annapolis, Maryland 21401

or

Info.SBE@maryland.gov with “Post-Election Ballot Tabulation Audit - Comments” in the subject line

​In response to a request from the Maryland General Assembly, SBE submitted a report on its post-election tabulation audit of the 2016 General Election (PDF)


1 The post-election audit methodologies tested during the pilot were 1) an independent, automated audit; 2) a ballot level audit applying Risk Limiting Audit principles; and 3) a fixed percentage audit.

2 This is calculated as the percentage of:  Absolute sum of all discrepancies in a contest
Total number of votes in a contest

3 For canvassing purposes, each early voting center is a separate “precinct.”   Each absentee canvass and provisional canvass is a separate precinct.