IST Seminar – Large Scale Data Analysis Challenges

Tuesday, February 24th
12:00 – 1:00pm
74 Jorgensen
(lunch provided)

“Large Scale Data Analysis Challenges”

Dan Meiron
Fletcher Jones Professor of Applied & Computational Mathematics and
Computer Science

JASON, a scientific advisory group, was asked by representatives of the Department of Defense (DOD) and the Intelligence Community (IC) to recommend ways in which the DOD/IC can handle present and future sensor data in fundamentally different ways, taking into account both the state-of-the-art, the potential for advances in areas such as data structures, the shaping of sensor data for exploitation, as well as methodologies for data discovery.

In this presentation we will examine the challenges associated with the analysis of large data and in particular compare DOD/IC requirements to those of several data intensive fields such as high energy physics and astronomy. The conclusion is that while DOD/IC data requirements are certainly significant, they are not unmanageable given  the capabilities of current and projected storage technology.

The key challenge will be to adequately empower DOD and IC analysts by matching analysis needs to data delivery modalities. At a very cursory level, we will examine some current approaches that could enable better information fusion. We’ll also propose various grand challenges that could be used to assess and prioritize future research efforts in data assimilation and fusion.