Joseph Lorenzo Hall1
School of Information, University of California, Berkeley
Electronic voting has faced increasing scrutiny since the disputed presidential race in Florida 2000. There has been a considerable amount of attention, to varying degrees, on issues such as election security, auditing, usability and accessibility. One area virtually devoid of attention to date is that of market and economic issues in voting systems. Information about market relationships between voting system vendors and their customers remains relatively unexamined.
The formal and informal relationships between election administrations and voting system vendors play a large role in shaping elections. The core of such formal relationships is voting systems contracts in which both vendors and election officials negotiate and agree to the terms of an initial purchase agreement and subsequent licensing, support, maintenance, training, etc. These contracts govern many of the activities in an election cycle dealing with voting technologies. For example, contract terms tightly control pre-election evaluation and certification, typically spelling out the environment and parameters around acceptance testing and logic and accuracy testing. In addition, these agreements and proprietary claims are often central to post-election disputes and audits.2
Voting systems contracts often restrict, in complex ways, activities and disclosure relating to voting technology. This can directly impact what a jurisdiction can and cannot do and what kinds of public communications it may make. Certain activities related to oversight, such as security testing, public disclosure, auditing and assembling mixed systems, can pose complex legal questions relating to contractual agreements and intellectual property.3 Unfortunately, this can often affect the degree of transparency and the public's perception of transparency surrounding controversial voting technology.
In previous work, we outlined dimensions of transparency in terms of access, oversight and accountability.4 That work focused primarily on the access dimension of transparency by examining the role for disclosed and open source software in electronic voting systems. This work examines contractual relationships between voting system vendors and election jurisdictions to catalog terms and conditions that might limit oversight of voting technologies.
In section 2, we talk briefly about related work. Section 3 describes the data set and how we conducted the initial analysis. In section 4, we discuss the results of the analysis and in section 5 we offer recommendations for transparency-facilitating terms and provisions. Finally, we offer our thoughts on how to extend this work in section 6.
There are other U.S. governmental contexts in which both protection of intellectual property--usually in the form of trade secrets--and transparency directly conflict.5 However, typically one or the other prevails. Levine has highlighted, in addition to the environment surrounding computerized election systems, two other examples where trade secrecy has thwarted access, oversight and accountability: security vulnerability disclosure and municipal wireless procurement agreements.6
Using examples from the domain of electronic voting, Jones has demonstrated how blanket prohibitions on the disclosure of security-related information, either in law or vendor-jurisdiction agreements, directly inhibit public oversight.7 The electronic voting context is one in which we expect the optimal solution to involve both protecting manufacturers' interests in trade secret protection as well as achieving a high level of public disclosure.
To date, there has been no in-depth substantive analysis of contractual agreements between voting system vendors and state and local election jurisdictions. There has been only one research project that we know of that made use of contracts as input into their analysis. In 2006, The Brennan Center for Justice at the New York University School of Law published an analysis that used cost and pricing data from a set of voting system contracts to model the upfront and ongoing costs involved with electronic voting systems.8
We hope that our analysis of contractual transparency barriers will further the use of voting systems contracts as data for future analyses. In this spirit, we are creating a ``voting systems contract portal'' hosted by the NSF ACCURATE center that will facilitate downloading and submission of voting system contracts.9
The data set for our analysis consists of 55 separate contracts between state and local election jurisdictions and voting system vendors.
We stress that this is not a representative sample from which one can draw generalizable quantitative conclusions.10 We employed a convenience sample to acquire contracts from some of the major markets for electronic voting systems (i.e., California, Ohio, Florida) as well as a number of contracts from smaller jurisdictions. When setting about to do this work, we were interested in a qualitative sample that, as a threshold matter, helped us determine the nature of barriers to oversight in voting system contracts. We quickly determined in our research design that a sampling strategy that aimed to make generalizable conclusions from a representative sample was not within the scope of a first analysis.
With these comments about generalizability aside, some statistics of our sample include:
It was a challenge to assemble this data set. Voting system contracts aren't necessarily public documents and often pieces of contracts are considered proprietary and confidential. For example, in communication with the California Voting Modernization Board (VMB), which administers state and federal funds for improving elections, we requested certain missing pieces of some California contracts. The Board could not give us exhibits A-B of Solano County's old contract with DESI, explaining:
``The VMB is in possession of Solano County's February 2003 Diebold contract exhibits A and B. However, these exhibits are identified as `Confidential Trade Secret Information,' and are therefore privileged and not for public disclosure.''14
Depending on a State's public records or open records laws, pieces of a contract that are considered proprietary and/or confidential may be exempt from disclosure.15 However, even when sections of a contract are missing, we can use tables of contents (when present) to give us clues about the content being claimed as proprietary and/or confidential. Typically, withheld portions of contracts are pricing related information.
We obtained most of the contracts in our data set from the California Voter Foundation, the Brennan Center for Justice at New York University School of Law, the California Voting Modernization Board and the Black Box Voting Document Archive. Other contracts were obtained individually through email solicitations we sent to various election-related email listservs requesting procurement-related materials.
After obtaining the contracts on paper, we scanned each contract into a PDF document at high-resolution (600dpi) and used Adobe Acrobat Pro's Optical Character Recognition (OCR) engine to translate the page images into text.16 We read the text of each contract to become qualitatively familiar with typical terms and conditions relevant to oversight transparency and developed a key of ``terms of interest'' for search-based extraction of blocks of text. The key was developed iteratively by marking up a few contracts from different vendors and locales and then finding common words of interest between them. The resulting key includes the following terms: ``confidential'', ``confid'', ``proprietary'', ``propr'', ``escrow'', ``trade secret'', ``trade'', ``secret'', ``source code'', ``source'', ``code'', ``benchmark'' and ``bench''.17 Finally, we printed out the blocks of text extracted from each contract and sorted them by vendor and then by date. The final analysis involved looking at these documents in order and manually comparing how they evolved over time in the context of basic information about the procuring jurisdiction.
Voting system contracts restrict copying, duplication, decompilation, reverse engineering, and preparing derivative works as well as other actions like translation,18 analysis,19 and even extend to training materials and ballots.20 Contracts consider it a breach of confidentiality if anyone else than the customer's ``employees, agents or contractors'' engage in these kinds of activities. And, complicating matters, confidentiality obligations can extend to information recorded in tangible forms (hardware, software, manuals, etc.) as well as oral communications and ``know-how'' obtained while interacting with the system.21
Certain types of information are explicitly excluded from being ``confidential'' in some contracts. Non-confidential information usually includes information in the public domain, information that the vendor discloses itself, information that becomes known without being misappropriated and information independently developed by the customer.22
While some of these provisions are typical of mass-market software licenses, other types of restrictions--such as limitations on the analysis of the voting system--are much more broad. Certainly, trade secrecy and other types of information protection are not usually troubling to a typical mass-market computer software customer. However, scholars have begun to question whether or not trade secrecy should be honored in applications involving governmental infrastructure such as electronic voting.23
Some contracts explicitly restrict output from voting systems. For example, ES&S has a standard term in the majority of its contracts that restricts copying or printing of output from any ES&S software:
Customer shall not [...] Cause or permit any copying, reproduction or printing of any output generated by the ES&S Software in which ES&S owns or claims any proprietary intellectual property rights (e.g., copyright, trademark or patent), including, but not limited to, any ballot shells or code stock.24This language was added to ES&S contracts in 2002 to cover ``any output'' and to specifically control ``ballot shells or code stock'', referring to blank ballots printed with timing marks for use with optical scanning systems.25
Meaningful oversight of electronic voting systems requires access to detailed data produced by the voting system. Outputs such as ballot images, raw vote data, audit and event logs, etc. are becoming increasingly important in litigating election disputes involving election technology as well as certification and evaluation of voting systems. The bulk of contracts in our data set do have explicit carve outs for permitted activities that have become necessary as part of election administration. Most of these contracts permit election-related and internal uses of proprietary and confidential information, archiving and backup of software products, copying to enable ``emergency restarting'' and even replacement of worn copies of software.26 While election administration offices may use this kind of detailed information internally, many of these offices do not have staff with the expertise needed to analyze these data to facilitate oversight. Thus, restrictions on disclosure of such output will complicate such activities if not stop them altogether.
In 2003, some ES&S contracts began to explicitly permit public demonstrations of voting machines and allow jurisdictions to print their own ballots27 or procuring the printing of their ballots from a firm other than ES&S.28
In a particularly interesting display of controlling the flow of information, San Bernardino's 2003 contract with Sequoia has a blanket, bilateral prohibition on public communications without both parties' written approval.
RELEASE OF INFORMATION. No news releases, advertisements, public announcements or photographs arising out of this agreement or SEQUOIA's relationship with COUNTY may be made or used without prior written approval of the COUNTY and SEQUOIA.29This provision would be more troubling if unilateral on behalf of the vendor, but in this context it appears that both parties to the contract felt it was in their best interests to require such a cumbersome chokepoint on information dissemination.
In some cases, contracts purport to grant trade secret protection to information that is not typically considered protectable in this way.30 For example, the unit prices that a vendor charges are typically claimed as proprietary in our data set. This is puzzling because, in most if not all cases, these numbers would be subject to budgetary disclosure provisions of the customer after the contract has been awarded.31 For example, in DESI's 2001 contract with Alaska:
It is expressly understood between the parties that [...] unit pricing constitute proprietary information the nature of which is a trade secret, and that disclosure of this information may place [DESI] at a competitive disadvantage.32
While many contracts limit claims for damages, in some cases, these limitations don't apply in the event of a breach of confidentiality:
Except for claims of personal injury and breaches of confidentiality obligations contained in this Agreement, CONTRACTOR and COUNTY liability for all damages shall not exceed the total value of this Agreement.33From an oversight perspective, this kind of a provision serves as a chilling effect on the customer. The jurisdiction may overly-protect election information, even that which might not run afoul of confidentiality requirements, out of fear of unknown and potentially massive damages.
In many cases, the physical location of the hardware or software and the specific computers on which the software runs are contractually restricted. One motivation for these kinds of restrictions is to contractually control the security environment in which the hardware and software operate. However, another equally compelling rationale for these geographic and platform-related controls is to prevent secondary markets from emerging where jurisdictions might rent or lease equipment and thus effectively compete against the vendor.
For example, vendors restrict the hardware on which their software may run:
Customer shall not, without DESI's prior written consent: [...] Use the DESI Application Software on any hardware other than the hardware identified in Exhibit A, Project Configuration Summary, the DESI Hardware on which it was pre-loaded by DESI, or other hardware for which DESI has granted its written approval.34In addition, DESI, Hart and Microvote have also restricted the physical locations or sites where their licensed software and/or hardware may be operated:
``Customer shall not [...] Use the DESI Application Software outside of Customer's jurisdiction [...].''35These types of restrictions can prohibit types of analyses that require the voting system to be examined in a specific environment such as a lab or problematic polling place. If the vendor objects and does not give its written authorization, provisions like these can effectively hold up location-specific activities.
As described above, third-parties are often excluded from being able to use or otherwise examine voting systems. Some jurisdictions have negotiated provisions that allow them to hire third-party programmers and operators to interact with their voting system, as long as those individuals are not employed by the vendor's competitors:
Diebold will allow the licensee to contract with outside individuals or firms to program using the GEMS software. The outside individual contractors will exclude individuals currently employed by the other election system vendors.36This type of provision balances the competitive concerns of the vendor and the desires of election administrators to hire the right expertise for a given job.
Contracts typically prohibit modification of their voting system hardware and software, without the written approval of the vendor, explicitly and through warranty limitations. However, in some cases any modifications have to be provided in source code form back to the vendor:
In the event Customers obtains approval to modify and/or enhance the Software Product, Customer shall provide [DESI] with the source code for the modifications and/or enhancements.37This kind of provision may discourage third-party commercial auditing of voting systems where the auditing firm writes custom code, database reports or source-code level analysis tools to analyze the system's operation.38
Finally, some contracts feel the need to clarify that voters or ``individuals participating in an election''39 are allowed to use and operate the voting system, but only in a manner according to the voter instructions for a system:
``Voters are also authorized to interact with the Sublicensed Software, in a manner consistent with voter instructions.''40It is notable that vendors and jurisdictions recognize that these agreements might be construed, without such provisions, to be so strict as to not allow voter interaction with the voting system. However, there is a wider sphere of uses that need to be recognized going forward, such as security evaluation, post-election auditing and election contests and litigation.
Most contracts include provisions that contemplate election officials' duties under Open Records or Public Records Laws. They also include a duty on behalf of the customer to notify the vendor of any such request within a certain amount of time. The period of time required between such notice and possible disclosure varies considerably from ``immediate''41 to ``as soon as possible''42 to ``as soon as public disclosure request is made''43 to ``promptly''44 to ``as much prior notice as reasonably practicable''45 to no less than 10, 15 or 20 business days.46 It is unclear why there is so much variation in these time periods.47 From an oversight perspective, it would be wise to tailor the time between notice and disclosure based on the information being requested and the time the request occurs in an elections cycle. For example, if it is a small, crucial request made before or immediately after an election, it should be disclosed as soon as possible and not subject to a delay in disclosure.
Contracts in our data set go so far as to declare the agreement itself as confidential. For example, the following provision appears in two ES&S contracts from different states:
[Confidential] Information includes the terms of this Agreement.48Considering how important contractual terms are in governing what a jurisdiction may or may not do with their voting system, other jurisdictions have explicitly negotiated language that allows full public disclosure of their agreement.49 From the 2006 contract between Yolo County, California and Hart:
Upon its execution, this Agreement (including all exhibits and attachments) shall be subject to disclosure pursuant to the California Public Records Act.50
Contracts in Florida explicitly limit the customer's liability due to open/public records disclosure:
The Supervisor shall not be liable for any damages suffered by Sequoia as a result of any disclosure of Sequoia's materials pursuant to [the Florida Public Records Act,] Chapter 119[, Florida Statutes].51This type of provision is similar, but in the opposite sense, to the lack of a limit on damages for confidentiality breach discussed in Section 4.1. This will tend to facilitate oversight through public records requests by ensuring that the customer will be shielded from any harm related to disseminating information about their voting system.
Many jurisdictions contractually or through regulations require vendors to deposit copies of source and object code with an escrow agent. The escrow agent is required to provide access to the escrowed code under certain conditions called ``release conditions''. Escrow agreements typically specify that source code shall be released to a jurisdiction if a vendor goes bankrupt, otherwise goes out of business or ceases to support or maintain a give product. In addition to these types of release conditions, the State of Ohio has negotiated a few more in its master contract:
(iv) Vendor makes the source code generally available to other users of the Licensed Materials (in which case Vendor shall make it available to the Secretary under similar terms and conditions); (v) Vendor is unable to correct a logic error or other bug in the software and such failure to correct constitutes an uncured breach of its obligations under Schedule E [the Software License Agreement]; or (vi) For purposes of temporarily auditing and/or testing the software source code held in escrow in accordance with the Escrow Agreement.52These additional provisions provide for access to source code if the vendor makes it available to other jurisdictions, if a vendor fails to correct ``bugs'' in the software or for temporary audit and testing purposes. If jurisdictions had the ability to patch bugs and fix vulnerabilities in the software that runs their voting systems, they could potentially have a powerful self-preservational recourse. There have been numerous cases of flaws in voting systems going ``uncured'' for many, many years; this type of provision would allow jurisdictions to claim access to voting system source code and contract for a solution.53 Of course, this would have to be done carefully but the idea is a promising one.54
Voting system source code is not usually provided to local jurisdictions. In the language of a recent contract between DESI and Alameda County, California, ``DESI does not provide its source code to Customers in the ordinary course of business.''55 and ES&S's standard contract includes specific language prohibiting use of source code:
The licenses granted in Section 2.2 do not permit Customer to use the source code for the ES&S Software Products. [...] The source code will remain the property of ES&S and may not otherwise be used by Customer.56
However, in contracts negotiated at the state-level we see evidence that access to source code is increasingly included in contract negotiations. For example, from the 2006 contract between Utah and DESI:
In addition, if requested, DESI will cooperate in order to enable a third party that is acceptable to the State to conduct an independent security review of its source code.57And from the 2004 master contract between DESI and the state of Michigan:
The Department of State or an authorized agent of the Department of State shall be able to obtain the software for purposes of analyzing and testing the software.58Considering the increasing importance of source code analysis in conducting pre- and post-election oversight and auditing of voting systems,59 having these provisions explicitly stipulated in the contract can ensure that such access is provided in a timely manner and in the form preferred by the jurisdiction.
In terms of access to source code for smaller jurisdictions we know of only one such agreement that provides for such access. Alameda County, California was recently able to negotiate a ``failsafe operation'' provision that provides for either a court or the California Secretary of State to call for a source code review if an unexplained discrepancy in vote data occurs during an election:
If there is an unexplained issue with votes being lost/added/changed during any election during the contract term and the California Secretary of State makes a determination that such unexplained issue requires investigation or if such a determination is so ordered by a State of California court, the County will have the election source code reviewed for malicious code by an independent third party mutually agreed upon by both parties with a Sequoia confidentiality agreement. The review will be commissioned by the County and, if so ordered by a court or the California Secretary of State, the cost borne by Sequoia.60
In other smaller jurisdictions we see either complete absence of these kinds of provisions or explicit prohibition of access during an audit.61 It is unfortunate that smaller jurisdictions don't seem able to negotiate access to source code as needed.62
Finally, also in the recent Alameda contract, we see the first contractual agreement to cooperate with future disclosed and open source software legislation:
In the event that ``open source code'' becomes a requirement of California law, Sequoia will work with the CA Secretary of State under the rules/regulations in effect at that time to comply with the law.63This kind of forward-thinking provision is undoubtedly a good idea considering the interest of California and Federal officials and legislators in increasing disclosure of voting system source code.
Some contracts explicitly prohibit publishing benchmark testing results of the voting software or sublicensed software products. For example, some ES&S contracts forbid publishing benchmarking tests of Oracle database software included in their product.64 Hart contracts directly restrict publication of benchmark test results of any software that they provide:
Client shall not publish any results of benchmark tests run on any Software.65Unfortunately, the term ``benchmark test'' is not defined in these contracts and could be construed to cover any kind of stress- or performance-testing, comparative or not.66
Earlier contracts in our data set contain mandatory software upgrade provisions. This is especially interesting given the scandal in 2003 surrounding uncertified software being installed in California.67 For example, from the DESI contract with Alaska in 2001:
[DESI] may provide the Customer with unsolicited error corrections or changes to the Firmware which [DESI], at its sole direction, determines are necessary for the proper operation of its APPLICATION SOFTWARE and/or tabulating equipment, and the Customer shall incorporate these corrections or changes into the System within ten (10) days of receipt from [DESI].68It is encouraging to see that these provisions ceased appearing in DESI contracts after 2003. Forced software upgrades, especially on a short time-scale and without any provisions for being close to an election, are extremely dangerous from a security perspective. From the perspective of oversight, forced upgrade provisions practically ensure that software a jurisdiction tested and evaluated months before would be suddenly essentially unknown to them.
Through the thicket of contractual issues in the last section, we distill a number of contracting principles that jurisdictions can use to better facilitate oversight.
All data processed by the Election System and any derivative works of such data produced by the Election System are instruments of service which shall be deemed the property of the COUNTY.70
There are a number of dimensions along which this work could be extended. In the Fall, we will extend the immediate findings of this work to produce exemplar contract provisions and best practices in vendor contract language that jurisdictions and voting systems vendors can work with to promote transparency. In addition, while this analysis was focused on oversight-inhibiting terms and provisions, there are other data elements, such as pricing information, that could provide a basis for other types of inquiry.
Research that uses a stratified sampling strategy would allow more generalizable quantitative conclusions. Stratifying along either a per-state dimension or by grouping jurisdictions according to important properties, such as population, would make possible determinations about how prevalent certain provisions are and how jurisdictional properties effect the result of negotiations.
A promising line of analysis would be in the longitudinal dimension. We fully expect there to be a substantial difference in the nature of contractual provisions in agreements executed before and after 2000 due to the increased scrutiny and competitive pressure in the post-2000 environment. Research based on a substantial sample of contracts before November 2000 would test longitudinal hypotheses.
This material is based upon work supported by the National Science Foundation under A Center for Correct, Usable, Reliable, Auditable and Transparent Elections (ACCURATE), Grant Number CNS-0524745. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
The author would like to acknowledge the assistance of Aaron Burstein, Kim Alexander, Larry Norden, Ray Martinez, Deirdre Mulligan, Dan Wallach, Dan Sandler, Bev Harris, and the computing staff of the UC Berkeley School of Information, without which scanning and OCRing thousands of pages of contracts would have been unthinkable.
This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.71)
Copyright © 1993, 1994, 1995, 1996,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999,
Ross Moore,
Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html -split 0 -show_section_numbers -local_icons -no_navigation jhall_evt07_html
The translation was initiated by Joseph Hall on 2007-06-28