Development of the Distributed Teravoxel Data System: Acquisition, Networking, Archiving, Analysis, and Visualization

Development of the Distributed Teravoxel Data System: Acquisition, Networking, Archiving, Analysis, and Visualization. Project Report.

Paul Dimotakis, Daniel Lang, Jan Lindheim, Santiago Lombeyda, John McCorquodale, Dan Meiron, James Pool and Mark Stalzer

Abstract: This project is entitled, “Development of the Distributed Teravoxel Data System: Acquisition, Networking, Archiving, Analysis, and Visualization,” or TeraVoxel, for short, as will be referred to below. It was conducted under NSF sponsorship, under a Major Research Instrumentation (MRI) grant, No. EIA-0079871. It began on 1 September 2000 with an initial project end date of 31 August 2003. Two no-cost extensions were granted and the project was completed on 31 August 2005. The overall project goals continue to be pursued, with further developments sponsored by other research and development projects that presently rely on the TeraVoxel resources. Motivated by investigations of flow turbulence, the Teravoxel project was designed to handle both laboratory and numerical-simulation data. As part of the laboratory-support, the capabilities of a recently developed, 1024 2 -pixel, low-noise, high dynamic range, KFS digital-imaging system were extended to higher sustained speeds (frame rate) and fast data-acquisition and -storage capabilities. Networking, archiving, analysis, and visualization infrastructure necessary to analyze turbulence data of the type acquired by the KFS imaging system, as well as from numerical simulations, and from other applications were also developed. The infrastructure that was developed supports the analysis and visualization of terascale experimental or simulation datasets and their comparison with theory and modeling, to help validate theories and simulation results. The infrastructure can, in particular, support transfer of terascale datasets across campus from the experimental facilities of the Graduate Aeronautical Laboratories (GALCIT) to the computational, storage, and visualization facilities of the Center for Advanced Computing Research (CACR). This infrastructure leverages and is integrated with other CACR resources. Present and anticipated CACR resources include high-performance networking, data archives, and successive generations of both Beowulf-class clusters and Hewlett-Packard’s high-performance shared-memory systems. These resources, combined with the proposed Teravoxel system, will build on CACR’s tradition of harnessing new technologies to create innovative large-scale computing environments.