Paths Forward for DANSE -- Distributed Data Analysis Architecture for Neutron Scattering Experiments

Compiled by Brent Fultz with help from Przemek Klosowski and Tim Kelley

The sense of the DANSE Organizational Meeting at Caltech on March 13, 2003 was that the DANSE software architecture offers a new approach to satisfy the data analysis needs of the SNS, and to satisfy the future needs of all North American neutron facilities. Important technical decisions need to be made on formats for data streams and support for component templates. These issues are solvable. Working (although primitive) prototypes of them were already demonstrated. The steps needed for DANSE to move forward as a project were discussed towards the end of the Organizational Meeting. It is hoped that this summary of the discussion will promote the broader dialog that is needed now.

There is an obvious constraint on the path forward for DANSE. No one individual has the scientific breadth to direct the software development in all fields of neutron scattering science. The software needs of different communities need to be determined by the communities themselves. Prioritization of tasks and directions within these subfields of neutron scattering needs to be done with care. Software that is overly specialized will not have a long-term impact. Nevertheless, DANSE offers the opportunity to elevate the scientific level of experimental work, and connect to modern theory. The importance of new or more sophisticated methods of data analysis, modeling, and theory, need to be assessed by the scientific sub-communities to optimize the scientific impact of the DANSE software development.

A simple structure for organizing the software subprojects is along the lines of traditional neutron meetings and review committees. These are listed below, together with opportunities for new developments in these fields. In parentheses are suggestions by some attendees of possible institutional interest:

Another way to organize the subprojects is along the lines of the scientific opportunities that cross several classes of instruments or measurements. This organization could follow scientific fields rather than instruments and techniques. If not an actual paradigm for project organization, some of these scientific topics below could provide infrastructural support for the projects listed above. Some suggestions presented at the meeting were:

More futuristic opportunities were also suggested. It is possible that others should be added to the list which includes:

Coherence and cohesion of the DANSE project requires interactions between the personnel. To some extent this will be the formation of a virtual facility with principal investigators plus scientific staff. Suggested parts of this infrastructure include:

The needs of the broad field of neutron scattering science can be covered with an instruments-based approach, a scientific subfield approach, or some combination of them. The path forward depends on a number of factors such as sources of funding, and interest at neutron facilities, universities, and other activities at national laboratories. These will be discussed at greater length after the DANSE project is proposed to the Experimental Facilities Advisory Committee of the SNS on March 27, 2003. Nevertheless, now is the right time for members of the neutron scattering community to think about what scientific software subprojects are important, and how they could be organized to provide the largest scientific impact.