New Research Directions for the National Geospatial-Intelligence Agency: Workshop Report (2010)

The topics of photogrammetry and remote sensing were tackled during the morning session of the first day of the workshop. Photogrammetry and remote sensing have experienced tremendous innovation over the last decade, with the development of new sensing technologies, improvements in spectral and temporal resolution, and advances in automated feature extraction techniques. The committee invited Dean Merchant and Russell Congalton to prepare white papers on photogrammetry (“Calibration of the Aerial Photogrammetric System”) and remote sensing (“Remote Sensing: An Overview”), respectively. Clive Fraser and Melba Crawford provided an overview and thoughts on future research directions in these topics. This section summarizes Dr. Fraser’s presentation, entitled “Spatial Information Extraction from Imagery: Recent Trends in Geomatics,” Dr. Crawford’s presentation, entitled “Advanced Sensing and Information Extraction: Synergies for Optical Sensing,” and the discussion that followed.

Photogrammetry, a subset of remote sensing, obtains accurate two- and three-dimensional coordinates and information for physical objects and the environment through the processes of acquiring, measuring, and interpreting photographic images. Geomatics is the discipline of gathering, storing, processing, and delivering geographic or spatially-referenced information, and it is largely concerned with calibration, measurement, and three-dimensional representation of objects. Traditionally, photogrammetric technologies and techniques were limited to photographic images. According to Dr. Frasier, globally, the field is expanding to include the interpretation and mensuration of imagery obtained from a wide variety of sensors and platforms, including multispectral and hyperspectral images, Light Detection and Ranging (LiDAR), and radar data. For example, photogrammetry now routinely encompasses digital aerial imaging systems, with a recent emphasis on the development of medium format, single and multi-sensor cameras; high-resolution satellite imagery with better than 0.5-meter resolution; airborne LiDAR with increasing pulse and scan frequency and full waveform recording; and airborne and spaceborne radar, including imagery and InSAR for digital elevation model (DEM) extraction.

In addition, there is rapid development of mobile mapping systems equipped with synchronized navigation and imaging sensors, such as digital cameras installed in stereoscopic pairs, LiDAR, or both. Low-cost photogrammetric systems using calibrated consumer-grade digital SLR (single lens reflex) cameras and inexpensive software are becoming more available and accessible, even to the non-photogrammetrist. In some respects, Google Earth, NearMap, and PhotoSynth also might be considered photogrammetry as these software systems provide two- and three-dimensional representations that are derived from imagery. However, according to Dr. Frasier, these software systems lack the metric integrity inherent in photogrammetry, raising issues of calibration, sensor modeling, georeferencing, and rigorous data fusion.

Dr. Frasier identified several key research challenges for photogrammetry and geomatics, including sensor modeling and georeferencing, feature extraction, and above all, increased automation of the spatial information generation process. Several photogrammetric operations are routinely automated, including interior and exterior orientation; control point recognition; elevation extraction with user input for refinement, checking, and correction; aerial triangulation with user input for ground control; orthophoto generation; some aspects of display and visualization; and three-dimensional scene generation in mobile mapping (Xiong and Zhang, 2010). However, the automated extraction of information (i.e., vector data) from photo-textured three-dimensional point clouds (such as those generated by terrestrial LiDAR scans) is still an area of ongoing research (e.g., Chen et al., 2007). Other areas of emerging research include calibration of complex multi-sensor cameras and the alignment of cameras to inertial measurement units (IMUs) and LiDAR; sensor orientation modeling using rigorous sensor modeling versus rational polynomial functions for multi-scene processing of high-resolution satellite imagery; automatic feature extraction, particularly for building extraction, topographic mapping, and utility mapping; monoplotting in the absence of stereo for close range three-dimensional object reconstruction via single images and a digital elevation model; forensic measurement with consumer-grade cameras (van den Hout and Alberink, 2010); image sequence processing and analysis; enhanced object modeling and classification via full waveform LiDAR; biomass estimation via radar and LiDAR technologies (Kellndorfer et al., 2010); and enhanced classification for feature extraction. Dr. Frasier also noted the need for research in data fusion, which is discussed in Chapter 3.

To summarize Dr. Frazier’s presentation, the principal challenge of photogrammetry and geomatics is centered on the automated generation of spatial information from multiple sources of imagery and ranging data generated by ubiquitous, integrated multi-sensor systems. While discussion on automating “feature extraction” highlights the challenges of automation, it is only a starting point and the solution is likely to involve a combination of research from traditional remote sensing as well as from the new cross-cutting disciplines. Higher spatial and temporal resolutions will be required to support a range of functions, such as change detection, monitoring, and GIS database update. Research needed to support these functions ranges from metric processing of remotely-sensed multi-sensor data to feature extraction and modeling.

In the second presentation, Dr. Crawford focused on the state-of-the-art in optical remote sensing technologies. Remote sensing is the science of acquiring imagery and information about an object or phenomena using sensors that are wireless or not physically connected to the object, such as from airborne or spaceborne platforms. Remote sensing technologies include high resolution panchromatic and multispectral sensors; hyperspectral sensors, which collect tens to hundreds of narrow spectral bands continuously across the electromagnetic spectrum; and LiDAR, which includes full waveform systems and photon counting techniques. Common

challenges in remote sensing include information extraction, data storage, and data product dissemination. Because hyperspectral imagery is collected over narrow bandwidths across a wide range of the electromagnetic spectrum, it is possible to extract detailed spectral patterns for a variety of rock and mineral types and for simple land cover and vegetation classification. These spectral patterns, in turn, can be used for image classification and chemical content analysis. Thus, hyperspectral imagery can potentially provide improved capabilities for atmospheric correction, characterization of targets of interest, pixel unmixing, anomaly detection, classification, and increased sensitivity to spatial and temporal variations. Improved atmospheric corrections are particularly important for multi-temporal or multi-sensor analysis, while improved sensitivity to spatial and temporal variations is important in parts of the world where ground-truthing is limited or impossible. The challenges of hyperspectral sensing, however, are the enormity and redundancy of the data sets, the significant number of parameters needed to extract information, and the sensitivity to spatial and temporal variations in signatures.

Dr. Crawford emphasized that machine learning focuses on the design and development of algorithms that allow computers to progressively learn behaviors based on empirical data, such as from sensor data or databases. Three machine learning techniques that show promise for extracting information from hyperspectral imagery are nonlinear manifold learning, semi-supervised learning, and active learning. Nonlinear manifold approaches, which reduce the dimensionality of the imagery via non-linear transformations, include local linear embedding (Roweis and Saul, 2000), isometric feature mapping (Tenenbaum et al., 2000), and the commonly used Kernel Principal Components Analysis (Scholkopf, 1998). Semi-supervised learning makes use of both labeled and unlabeled data for training to generate a classification, and typically assumes that the labeled and unlabeled samples are from the same population. The unlabeled samples are used primarily to recover under-represented characteristics of labeled samples. Semi-supervised approaches include self-learning with ML classifier (Jackson and Landgriebe, 2001); Transductive Support Vector Machines (SVM) (Bruzzone et al., 2006); and semi-supervised SVM (Mingmin et al., 2007). Lastly, active learning mines the unlabeled data for information and interacts with the classifier to construct the training pool for supervised and semi-supervised learning.

In summary, Dr. Crawford outlined future research opportunities for advanced optical remote sensing, including interdisciplinary research in data exploitation, sophisticated visualization techniques and integration with data analysis, new computational paradigms for analysis and modeling, sensor integration and sensor web applications, and integration of advanced optical sensor data with three-dimensional and four-dimensional GIS functionality. Critical challenges include the focus on traditional data sources and methods of analysis, the gap between research and operational missions, and the need to train GEOINT professionals.

Working Group Reports. Working group reports on photogrammetry, geomatics, and remote sensing indicated that these fields are moving in the direction of four-dimensional mapping, including time, with the goal of achieving the ability to search for and analyze events and scenarios. This is a departure from the traditional use of sequential rectified images to detect and locate changes on the landscape. Workshop participants remarked on the critical need for hyperspectral and LiDAR data and on the growing importance of data acquired from non-traditional platforms, such as networks of spatially distributed sensors (i.e., sensor networks and sensor webs), unmanned aerial vehicles (UAV), drones, “small satellites,” consumer-grade cameras, and cell phones. For example, cameras, sound recording devices, and inertial mapping

systems carried by individuals are being used for real-time mapping of interior spaces. The amount of data produced by these systems, however, can be significant and thus expensive and time-consuming to transmit from the point of acquisition to the point of processing. Therefore, constant data streams will need to be processed and georeferenced closer to the data acquisition system to ensure useful information in near real-time. More comprehensive metadata also will become available.

Workshop participants also noted that remotely sensed data needs to be integrated with other data sets, such as geographic information systems (GIS) data layers, data from metric and non-metric technologies, socio-economic data, cultural information, contextual data, and time series. Text information, for example, can be linked with remote sensing data to aid classification and to improve event and scenario recognition. Change analysis can be enhanced beyond the process of measurement and classification to model dynamics, behavior, and prediction. Atmospheric impacts also need to be exploited as signals.

Blending of information and integration of open source data provide new methods for information fusion. However, incorporation of these various kinds of data results in data of mixed type and unknown quality. Participants focused on data quality issues that will need to be addressed—including reliability, quality assurance and control, system calibration—with a more comprehensive use of supporting environmental information. Uncertainty and error need to be integrated into multi-sensor fusion models and methods of information extraction and analysis. Multiple sources of uncertainty, sensor errors, data confidence, and models (empirically vs. theoretically-based) need to be characterized. Advanced statistical estimation, automation, modeling and data processing, numerical methods, and optimization techniques also can be incorporated. State-of-the-art algorithms can be better utilized.

Workshop participants suggested that these complex problems will require new strategies that are interdisciplinary in nature and that incorporate multi-scale, multi-temporal, and multi-resolution data integration and analysis. Situationally-aware analysis tools will need to be tailored for specific end purposes. The infrastructure required to handle the massive volumes of data will be equally important, such as data storage, compression, distribution, and throughput to the analyst. More tools, better knowledge-based methods, visual analytics, metadata generation, process automation, and data mining for a specific sensor can augment the information flow to the image analyst. The analyst will require more than just imagery and knowledge of the physical landscape, including data on the dynamics and social environment.

The five traditional NGA core areas are being augmented with the blending of the fields of computer science, statistics, electrical and computer engineering, geodesy, geography, and bioinformatics.

The committee invited Robert McMaster, Dru Smith, and May Yuan to provide an overview of cartography, geodesy, and geospatial analysis, respectively, and to offer their thoughts on future research directions. This section summarizes their presentations and the discussion that followed. The discussions were informed by white papers written by Robert McMaster (“Cartographic Research in the United States: Current Trends and Future Directions”), Lewis Lapine (“A Brief History of Satellite Geodesy - October 4, 1957 to Present”), and Luc Anselin (“Geospatial Analysis”).

The first keynote address, given by Dr. McMaster, focused on “Trends in Cartographic Science.” Dr. McMaster revisited research priorities outlined by the University Consortium on Geographic Information Science in the late 1990s. The long-term challenges were listed as spatial ontologies, geographic representation, spatial data acquisition and integration, the use of remotely acquired data in GIScience, scale, spatial cognition, analysis and modeling of space-time data, dealing with uncertainty, and visualization. Other challenges included seeking GIS’s role within society, and geographic information engineering (i.e., distributed computing, the future spatial information infrastructure, data mining and knowledge discovery). Pressing short-term challenges were listed, and among these geocomputation and geographic information security were highlighted.

Dr. McMaster then focused his attention on three important themes in current cartographic research: scale and generalization (i.e., the process of simplifying information on a map, such as the boundary, especially as the scale of the map becomes smaller); geographic visualization; and public participation mapping (Elwood, 2006; Sieber, 2006; Tulloch, 2008) and volunteered geographic information (Goodchild, 2007; Elwood, 2008; Flanagin and Metzger, 2008). Research questions of interest included understanding how scale affects human perception, how scale can be measured and characterized, how to automate scale change in mapping systems, and how scale and scale change affect information content, analysis and conclusions about spatial patterns and processes. Under visualization, collaborative systems, information visualization, spatialization, multivariate mapping and animation were seen as in need of basic research. Lastly, the social implications of GIS were discussed, given a recent rise in participatory mapping and volunteered geographic information collected via on-line mapping systems.

The second keynote, presented by Dr. Smith, entitled “An Pptimist’s 20-Year Look Ahead at Geodesy and Geophysics,” compared the predictions formulated by leading past reports on geodesy (i.e., Whitten, 1963; NRC, 1985; Sanso, 2003; IVS, 2006; Plag and Pearlman, 2009; Wanninger, 2008) with the current state of the art. Significant innovations in geodesy raised in these historical glimpses of the future include satellite geodesy, an earth centered reference frame, distancing by laser, and the measurement of gravity potentials. A 1985 NRC report (NRC, 1985) raised the advent of the Global Positioning System, solutions to changes in geodesic measurements, merging absolute and relative geodesy, and improving inertial systems. In 2001, GPS expert Richard Langley forecast that by 2084, GPS would be capable of 1 mm accuracy in seconds, for $10 by a wristwatch (Wanninger, 2008). A 2003 International Association of Geodesy report forecast that a global reference frame would be available accurate to millimeters horizontally and centimeters vertically, that the earth’s geodetic sub-systems would be modeled as interacting, and that geodesy would involve combining massive data sets. Lastly, a 2009 study (Plap and Pearlman, 2009) noted that geodesy would need to meet the needs of global change with continuous operational monitoring systems, while integrating new imagery, such as gravimetry, with point-based data in GIS. Many of these forecasts have now been realized, but some remain elusive.

Dr. Smith’s own forecasts of the geodetic systems of the future included pervasive Global Navigation Satellite Systems (GNSS) with sub-meter instantaneous precision, widespread use of EGM08 and a world height system in military theaters, improved gravimetric imaging, and drastically more accurate atomic clocks. These would support such applications as sea-level rise monitoring, navigation in the Arctic, earthquake and tsunami warning systems, and non-GNSS systems for navigation and positioning indoors and underground.

Based on this assessment, Dr. Smith identified five main themes for research in geodesy and geophysics:

1. how to get optimal fusion of super-massive quantities of data;

2. how to fold cutting edge new technologies into operational use;

3. how to get exterior positions to centimeter (and in the future millimeter) accuracy in real time;

4. how to set up a long term Earth monitoring service, with geodesy as the foundation; and,

5. how to deal with a public that is increasingly “position capable,” but ignorant of geodesy.

The third keynote address, delivered by Dr. Yuan, explored “Spatial Integration and Spatiotemporal Inspiration.” Dr. Yuan characterized the state of the art in GIS as reflecting spatial integration, both horizontally through conflation and vertically through overlay. The challenges to spatial integration were seen as developing a spatially integrated framework of data, models, and decision-support systems; doing vertical as opposed to horizontal integration; and analysis and modeling (Hornsby and Yuan, 2008). Current solutions include mash-ups and participatory systems, and geosensor webs such as WeatherSense. The future was viewed as “spatiotemporal inspiration.” Specific challenges included dealing with space-time memory and space-time clues, space-time ordering and spatiotemporal language, cyber GIS and real-time applications, and space-time and geographic dynamics (Pultar et al., 2010). Dr. Yuan contrasted the geographic measurement framework, the GIScience relational object model of geographic space, and her own model of geographic dynamics, which includes activities, events and processes. Examples from atmospheric systems were used as illustrations. Particular emphasis was given to the concept of a “narrative GIS,” or the role of GIS as a compiler for spatial event sequences seen as “stories.” Future research needs in spatiotemporal inspirations would require the GIS to support moving from forms to processes to narratives consisting of linked event sequences. Narratives can be transformed to possibilities (e.g., scenarios), which can be characterized by metrics to assess and create similarity gradients (e.g., the difference in a situation from the average or from last year), which in turn can guide prediction and forecasts based on the probabilities.

Working Group Reports. In addition to reiterating a number of themes raised in the keynote presentations, the breakout groups formulated several topics in need of further research. With respect to cartographic science, the need to improve the speed of map presentation was noted, moving beyond tile-based mapping and Mercator projections to continuous-scale mapping. Scale itself needs to be transformed from a purely cartographic focus to include semantics and temporal dimension, a theme echoed in the discussion of GIS and geospatial analysis. Similarly, the incorporation of volunteered geographic information was raised as an important issue for both cartographic research and GIS. In terms of future technology, the extension of interactive cartography was suggested, by considering the cognitive effectiveness of geospatial technology to include eye tracking, brain sensing and the use of other body sensors.

Workshop participants’ discussion of geodesy focused on the importance of global navigation satellite systems, especially GPS, the move towards ubiquitous geopositioning (e.g., multiple receivers, phone, navigation), and the need to integrate GPS into all aspects of geospatial technology. Ensuring user proficiency in the proper use and interpretation of positioning data is equally important. In addition, gravimetry, especially to improve our

understanding of time-dependent gravity, is an exciting new area of research. A third issue raised was the need to expand the ground-based continuously operated reference system (CORS), which provides GNSS data consisting of carrier phase and code range measurements in support of three-dimensional positioning, meteorology, space weather, and geophysical applications throughout the United States, its territories, and a few foreign countries. Impressive progress has been made in geodetic accuracy, but improvements are needed in areas where GNSS does not work, such as under tree canopies, in urban and natural canyons, and underground and underwater. With respect to precision, participants expressed the desire to establish a geodetic reference frame at a sub-millimeter accuracy level and the next generation of positioning instrumentation and inertial navigation systems stable to the centimeter level over time. The importance of high performance computing to support such an effort was noted, for example as it relates to issues of reliability and latency associated with the handling of massive quantities of data.

Regarding GIS and geospatial analysis, several breakout groups stressed the importance of the temporal dimension. A truly comprehensive space-time GIS and geospatial analytical framework remains to be developed. A second important theme was the incorporation of heterogeneous sources of information into a GIS. This includes information from unstructured sources (e.g., text), social and knowledge domains with GEOINT, and volunteered geographic information. A core concern and research need in this respect is the assessment and representation of the quality of that information from authoritative and non-authoritative sources (specifically benchmarking); the visual representation of quality, reliability and confidence; and their interpretation by the user. The concept of social mapping was noted, as was the need for integration of geospatial and social networks.

A third theme centered on the concept of the GIS narrative and its expansion to multiple levels of explanation, involving the need to conceptualize complex information into a story line. A further understanding of how to work with the narrative framework is needed as well. Related to this is the overall communication of geospatial issues (both static and dynamic) and their visualization. Fourth, in terms of geospatial analytical methodology, the potential for game-based analytics was noted, as well as the need for automated service and workflow discovery to enable automatic tool application. The last two themes that emerged in several discussion groups were the need to develop interdisciplinary training and education to generate the needed workforce and an assessment of the current organization of NGA along the five traditional disciplines. Some concern was voiced that the latter may hamper interdisciplinary and collaborative investigation, which were viewed as key to obtaining significant research advances.

Specific topics outlined by the working groups are summarized in Appendix E.

The topics of forecasting, participatory sensing, and visual analytics made up the morning session of the second day of the workshop. Three keynote presentations set the stage for further discussion. Antonio Sanfillipo gave the initial keynote entitled “Technosocial Predictive Analytics: Bridging the Gap between Human Judgment and Machine Reasoning.” Forecasting is

an operational research technique used to anticipate changes, outcomes, trends, or expected future behavior of a system using statistics and modeling. For example, geospatial forecasting can be used to anticipate the impact of climate change and energy consumption on the power grid and critical infrastructure. In his presentation, Dr. Sanfillipo identified three research challenges: (1) to combine diverse elements of knowledge to promote model fidelity and reliability; (2) to link across diverse modeling algorithms to use the right tool for the right job and to leverage legacy models; and (3) to stimulate creative reasoning through collaborative work with interoperable models.

Cyrus Shahabi gave the second presentation, entitled “Participatory Urban Data Collection: Planning and Optimization.” Participatory sensing tasks everyday mobile devices, such as cellular phones, to form interactive, scalable sensor networks that enable the public and professionals to gather, analyze, share, and visualize local knowledge and observations. Dr. Shahali’s key issue was the optimal placement of the sensors that people would use to collect geospatial data, and the major challenge was planning and optimization to enable the collection of useful data from a broad group of untrained participants. In addition, the speaker raised concerns regarding privacy and trust between volunteers and project organizers, which will need to be addressed. Several successful project examples were described, including participatory texture documentation (Banaei-Kashani et al., 2010).

David Ebert offered the third keynote presentation on “Proactive and Predictive Visual Analytics.” Since the publication of Illuminating the Path: The R&D Agenda for Visual Analytics (Thomas and Cook, 2005), the field of visual analytics has grown significantly. Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces. It is a multidisciplinary science, drawing upon the domains of knowledge discovery, cognitive and perceptual sciences, interaction science, decision sciences, geospatial analytics, scientific computing, graphics and visualization, statistical and information analytics, and data management. Dr. Ebert stated that visual analytics needs to provide an interactive, integrated discovery environment that can be user- and perceptually guided, balancing human cognition and automated computerized analysis to amplify human cognition. The key challenges for proactive and predictive spatiotemporal visual analytics mentioned by Dr. Ebert were: (1) making computational simulations and statistical analysis interactive; (2) integrating and analyzing massive and streaming multi-scale data for cross-scale visual analysis and to simulate multi-scale, multi-system, multi-source interactions; (3) creating seamless natural interaction with and multivariate, multidimensional representations of spatiotemporal environments, and also to develop new temporal and spatiotemporal directable and adaptive predictive models; (4) developing intuitive visual analytics for uncertainty and time; (5) integrating computer-human visual cognition environments to create interactive planning and decision-making environments; (6) adapting spatiotemporal analytics to integrate and perform with user-specified knowledge, context, constraints and boundaries; (7) adjusting algorithms to enable mobile context-sensitive analytics and balance local-remote distribution of work; and (8) adapting visual analytics to the user, environment, and sources of data. A recent suite of nine articles published in the International Journal for Information Visualization in Fall 2009 describe the first five years of progress in this science, including early success stories.

Working Group Reports. The working group participants noted that challenges in forecasting include predicting human behavior; the lack of theory; the prior lack of integration with geospatial data; and the incorporation of spatial analytic methods, validation techniques, and

space-time issues into predictive models. Participatory sensing was thought to be understudied in geospatial data. Challenges identified by participants include preserving individual privacy, dealing with unstructured and varying quality data, exploiting social networking tools, capturing the context of data, and encouraging participation. Visual analytics development was thought to require developing a repeatable body of knowledge within the geospatial field, which includes interactivity and visualization. New ideas could be derived by looking at aggregates and across scales. Additional challenges mentioned were dealing with space-time and massive data sets; developing metaphors, games, and animations for understanding patterns; improving modeling and simulation; and using high-performance computing and the proper depiction of data quality and error uncertainties.

Specific topics outlined by the working groups are summarized in Appendix E.

The committee invited Haesun Park and Kathleen Carley to provide an overview of data fusion and human terrain analysis, respectively. This section summarizes Dr. Park’s presentation, entitled “Data and Visual Analytics for Information Fusion,” Dr. Carley’s presentation, entitled “Human Terrain—Assessing, Visualizing, Reasoning, Forecasting,” and the discussion that followed. Data fusion is the aggregation, integration, and conflation of geospatial data across time and space with the goal of removing the effects of data measurement systems and facilitating spatial analysis and synthesis across information sources. Human terrain is the creation of operational technologies that allow modeling, representation, simulation, and anticipation of behaviors and activities of both individuals and the social networks to which they belong, based on societal, cultural, religious, tribal, historical, and linguistic knowledge; local economy and infrastructure; and knowledge about evolving critical events.

In her keynote presentation, Dr. Park made a distinction among early, middle, and late data fusion options. These are sometimes termed object, situation, and threat fusion, respectively, in an intelligence context (Liggins et al., 2009). Early fusion is computer intensive, lacks engagement of the domain knowledge, and requires a common feature representation. Late fusion reduces features to concept scores or weights, using voting and rank aggregation, and makes interrelations hard to interpret. Kernel functions were discussed as a means to achieve early fusion (Munoz and Gonzales, 2008). Examples of fusion of breast cancer data, images and audio data, photographs and fingerprints, and handwritten text were presented. Methods of use in discriminating among instances and identifying new instances were discussed. It was noted that for effective fusion, breakthroughs in mathematics, statistics, algorithms, software, and systems would be necessary. According to Dr. Park, the key future challenges are related to data (e.g., volume, lack of structure, noise, heterogeneity, and space-time nature), computability (e.g., few algorithms are capable of handling massive, complex data sets) and links to real-time interaction, and visualization methods for exploring fused data.

Dr. Carley’s keynote presentation defined human terrain as an actionable description of the population, its culture, and its points of influence from a geo-temporal context. Areas of overlap include international affairs, geopolitics, conflict modeling, culture modeling, peacekeeping, humanitarian and relief operations, consumer behavior, and dynamic network analysis. A goal of human terrain work is to create operational technologies that allow modeling, simulation, and anticipation of behaviors and activities of both individuals and the organizations

to which they belong that are sensitive to the geo-temporal-cultural context (Keller-McNulty et al., 2006). These methods make extensive use of network science and theory. Data comes from text, gazetteers, maps, and other sources. Often techniques examine links between elements, such as members of social groups or tribes, or identify critical points in networks, such as sequences of ports-of-call for vessels or vulnerable points in network traffic (Prietula et al., 1998). Many different visual display methods are used to examine the networks for these points. Challenges stated by Dr. Carley were getting social scientists to use the many available tools, access to geo-tools such as open-source geobrowsers, combining spatial and social models, the incompleteness of social data and the lack of a central theory, and the accuracy and reliability of self-reported data.

Working Group Reports. In the working group discussions, research issues of interest included the integration and fusion of social data, especially given the modifiable area unit problem and ecological fallacy; the lack of accurate GPS point traces on individuals, and hence reliance on census and other data sources; the need to integrate highly disparate data on economy, sociology, transportation, anthropology, ethnicity, religion, culture, and history; and the large differences in data certainty and reliability.

Fusion challenges identified by the working group participants were integrating data across spatial scales; dealing with semantic interoperability; conflation; dealing with sensors with different resolutions or spatial frameworks; and integrating at the data, information, and knowledge levels. Workshop participants also indicated that data fusion needs are still critical in remote sensing and are complicated by sensor networks. Sensor level fusion, using techniques such as support vector machines and Bayesian modeling, seem to be making advances, but the fusion of hard and soft data is still an unsolved problem. The role of humans as agents of data fusion was thought to be in need of study.

Specific topics outlined by the working groups are summarized in Appendix E.