Brian A. Jackson
Since the terrorist attacks of September 11, 2001, the sharing of intelligence and law enforcement information has been a central part of U.S. domestic security efforts. Though much of the public debate about such sharing focuses on addressing the threat of terrorism, organizations at all levels of government routinely share varied types of information through multiagency information systems, collaborative groups, and other links. Given resource constraints, there are concerns about the effectiveness of information-sharing and fusion activities and, therefore, their value relative to the public funds invested in them. Solid methods for evaluating these efforts are lacking, however, limiting the ability to make informed policy decisions. Drawing on a substantial literature review and synthesis, this report lays out the challenges of evaluating information-sharing efforts that frequently seek to achieve multiple goals simultaneously; reviews past evaluations of information-sharing programs; and lays out a path to improving the evaluation of such efforts going forward.
Improved Methodologies Are Needed to Measure the Value of Information-Sharing and Fusion Efforts
•The lack of literature on evaluating information sharing, coupled with passionate arguments both for and against the value of such efforts, has produced a stunted policy debate that is insufficient to support reasoned and reasonable tradeoffs among these programs and other ways to pursue the goals they are designed to advance.
•With a clearer framing of the evaluable goals sharing programs are pursuing, data on organizational outcomes can be linked to different ways of assessing the "dosage" of exposure to information sharing at different levels.
•New analytic techniques that enable matching of individual users and comparison of outcomes at a very disaggregated level appear particularly promising for assessing existing initiatives.
•Systematic approaches like "but-for" analyses and structures that tie sharing to the outcomes it is designed to achieve provide paths toward improved measurement of the value of information-sharing and fusion efforts.
Recommendations
•Methodologies to measure the value of information-sharing and fusion efforts should focus on outcome measures.
•Making analytic investments to improve methodologies for assessing the value of information-sharing systems is important if future decisions about the preservation, maintenance, or expansion of such systems is to be based on objective data instead of assumptions and anecdotal evidence of their effects and value.
The information has been taken from http://www.rand.org