Skip to main content

Currently Skimming:

2 A Framework for Evaluating Information-Based Programs to Fight Terrorism or Serve Other Important National Goals
Pages 44-66

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 44...
... For purposes of this framework, this chapter describes all of these, together with the people who operate them, as information-based programs because they have in common their reliance on information about individuals. This chapter proposes a framework for evaluating and deploying technologies, programs, and systems that rely on personal data to prevent terrorism or to serve other important national goals.
From page 45...
... Building on the work of these prior efforts and informed by the members' experiences and research, the committee designed a framework to guide public officials charged with making decisions about the development, procurement, and use of information-based programs. Its purpose is not to impose bureaucratic compliance requirements, but rather to assist well-meaning people at every level of government to do their jobs better, to enhance their effectiveness in countering terrorist threats, to facilitate the wise and timely implementation of new programs, to invest limited government resources wisely, and to ensure that basic American values are not compromised when doing so.
From page 46...
... For any potential program, policy makers will have to exercise sound judgment in deciding whether the program is sufficiently effective and sufficiently protective of privacy to warrant proceeding with it, although such judgment should be undertaken after the framework has been applied rather than before. The questions posed by this framework should be asked not only of all new information-based programs, but also of existing programs today, at regular intervals in the future, and any time that a program is to be altered or put to a different use, to ensure that scarce resources are invested wisely; tools are used appropriately, lawfully, and consistently with societal values; and the best protection is pursued for national security and civil liberties.
From page 47...
... For example, grounding all airplanes would be a highly effective technique for preventing terrorist bombings of airplanes in flight, but it would not be a workable solution because it would also keep millions of law-abiding passengers from flying. As this example suggests, ineffective or overly broad programs often create significant side effects that extend far beyond the immediate impact on the data subjects.
From page 48...
... They are intended to ensure that the nation invests its human, technological, and financial resources wisely. They should be addressed before a new information-based program is procured or deployed and, as appropriate, at regular intervals during the development and use of such a program.
From page 49...
... Experimental science, and much of engineering as well, generally involves a logical progression from theory to simulations to laboratory tests, to small-scale field tests, to larger scale tests. In the rush to find quick responses to pressing national security concerns, there is a natural tendency to want to skip one or more of these phases, but the hundreds of millions of dollars wasted on systems that did not go through appropriate experimentation and subsequently did not work suggest that such omissions seldom pay off.
From page 50...
... 9. The information-based program should provide for appropriate data stewardship, a term that refers to accountability for program resources being used and protected appropriately according to the defined and authorized purpose.
From page 51...
... When such a system uses personally identifiable information or otherwise affects privacy, the documentation should be examined by an entity, such as an independent scientific review committee, that is capable of evaluating the scientific evidence of effectiveness outside the agency promoting the new system.
From page 52...
... There are also practical, utilitarian reasons for concern about values. Promising antiterrorism systems may be derailed, even ones well within existing law, because they so offend popular and political understandings of privacy that go beyond existing legal requirements.
From page 53...
... They should be addressed by agency officials before a new information-based program is procured or deployed and, as appropriate, at regular intervals during the development and use of such a system. The committee also believes that the criteria should be useful to judicial and congressional officials as they evaluate new and existing programs and determine the boundaries of the nation's laws protecting privacy and other civil liberties.
From page 54...
... There must be in place a process for identifying the frequency and effects of false positives and for dealing with them (e.g., reporting false positives to developers to improve the system, correcting incorrect information if possible, remedying the effects of false positives as quickly as practicable) , as well as a specific locus of responsibility for carrying out this process.
From page 55...
... Moreover, the proliferation of digital data and dramatic reductions in the costs associated with sharing and storing data mean that even irrelevant data are routinely retained by the government indefinitely. Giving new force to minimization requirements is essential to avoiding the situation of government maintaining ubiquitous data records that threaten to invade personal privacy and overwhelm efforts to use data effectively to enhance security.
From page 56...
... The deployment or use of any informationbased program that relies on sensitive personally identifiable information, personally identifiable information collected surreptitiously, personally identifiable information that has been obtained from a third party without individual consent, or personally identifiable information that is being used for a purpose that is incompatible with that for which it was originally collected should be conditioned on an appropriately specific authorization from a source external to the information-based program.6 Typically, this would be authorization by an appropriate court (federal Article III, Foreign Intelligence Surveillance, or state) , but Congress may provide for other forms of external authorization.
From page 57...
... The committee intends the entire framework proposed in this chapter to be useful to policy makers in outlining issues to be addressed through legislation or regulatory policy, as well as in proposing specific steps for ensuring that the nation fights terrorism effectively and consistently in accord with its core values. However, the breadth and variety of information-based programs, as well as the constantly changing capacity of technology, make crafting legislation governing those programs and protecting civil liberties a difficult task.
From page 58...
... Are there robust systems in place to identify errors, such as false positives, use them systematically to improve information-based programs, and provide rapid, effective redress to affected individuals?
From page 59...
... 3. Is there a sound experimental basis for the information-based program and each of its components?
From page 60...
... • Are there adequate guarantees of the information's validity, provenance, availability, and integrity? • Are the data easily compromised or manipulated so that the system can be defeated?
From page 61...
... 8. Is there a process in place for identifying the frequency and effects of false positives and for dealing with them (e.g., report ing false positives to developers to improve the system, cor recting incorrect information if possible, remedying the effects of false positives as quickly as practicable)
From page 62...
... 15. If the information-based program relies on sensitive personally identifiable information, personally identifiable information collected surreptitiously, personally identifiable information that has been obtained from a third party without individual consent, or personally identifiable information that is being used for a purpose that is incompatible with that for which it was originally collected, have its deployment and use been conditioned on authorization from a source external to that in which the information-based program will exist, and have they been approved by an external authority (e.g., an appropriate court or other authority)
From page 63...
... • Have the purpose for which the data were collected, their age, and the conditions under which they have been stored and protected been taken into account when determin ing whether the proposed information-based program is appropriate? • If data are to be used for purposes that are inconsistent with those for which they were originally collected, has the agency specifically evaluated whether the inconsistent use is justified and whether the data are appropriate for such use?
From page 64...
... • Is that demonstration based on scientifically valid criteria? • Are there credible processes in place to measure effective ness and to ensure continual assessment of effectiveness and efforts to improve effectiveness?
From page 65...
... 6. Redress • Is there a process in place for identifying the frequency and effects of false positives and for dealing with them (e.g., reporting false positives to developers to improve the sys tem, correcting incorrect information if possible, remedying the effects of false positives as quickly as practicable, and so on)
From page 66...
... • Is the information-based program audited not less than annually to ensure compliance with the provisions of this framework and other applicable laws and regulations? • Are the results of ongoing assessment documented?


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.