October 8, 2001
Eighty-two new servers and significant improvements in the overall
computing and networking infrastructure will be added this fall
to the PDSF (Parallel Distributed Systems Facility), a large Linux-based
computer cluster that is currently operated as a partnership between
three divisions of the Lawrence Berkeley National Laboratory: Nuclear
Science, Physics, and the National Energy Research Scientific Computing
Center (NERSC). Computing power will be expanded by 57 dual-processor
nodes, and storage capacity will be tripled.
The PDSF serves the data-intensive computing needs of international
high energy and nuclear physics experiments at the world's most
powerful accelerator centers -- including the AGS/RHIC complex at
Brookhaven National Laboratory (STAR, E895), CERN (NA49, ATLAS,
ALICE), and Fermilab (CDF, E871) -- as well as neutrino detectors
in Canada (SNO) and Antarctica (AMANDA), and other experiments that
are expanding our knowledge of the universe.
"The large computing and storage capabilities of the PDSF play
an essential role in helping researchers extract the physics from
the terabytes of data that they produce," said Lee Schroeder, Director
of Berkeley Lab's Nuclear Science Division. "For example, the STAR
experiment is producing descriptions of nuclear collisions at the
highest heavy ion collider energies ever achieved, and SNO's first
physics results are helping to solve the 'solar neutrino problem'
and provide further evidence for neutrino oscillations. These great
physics results were made possible by the dedication of the NERSC/PDSF
staff and their close collaboration with the research teams."
Projects that use the PDSF provide support for expansion and upgrades
of the system. Currently the Berkeley Lab Nuclear Science and Physics
divisions and the NERSC Center cover the costs of four NERSC Division
employees who provide system administration and user support. The
sponsoring divisions within the DOE Office of Science are Nuclear
Physics, High Energy Physics, and Mathematical, Information, and
Clusters are assemblies of commodity computers designed and networked
to operate as a single system. The PDSF currently consists of 281
processors with a theoretical peak processing capacity of 155 gigaflop/s
and a total storage capacity of 7.5 terabytes. When the current
expansion is completed in October 2001, the PDSF will have 395 processors,
and the number of disk vaults will grow from 15 to 40, with a total
24 terabytes of shared storage.
Each new node will have two 1-gigahertz Pentium III CPUs, with
1 GB of memory per CPU. Gigabit Ethernet networking for the high-bandwidth
compute nodes will enable the PDSF to run MPI jobs that require
up to 50 nodes (100 processors). In addition, Gigabit Ethernet for
the disk vaults will make it possible to take advantage of the server-side
performance improvements of the 2.4 Linux kernel.
The STAR Collaboration is the largest user of the PDSF and is making
the largest investment in its expansion: 16 disk vaults, 20 compute
nodes, and 34 high-bandwidth compute nodes. The PDSF has become
such an integral and indispensable part of the project's data analysis
that they have changed the standard acknowledgment for all their
publications to recognize NERSC's support.
Other research groups contributing to the current PDSF expansion
are the CDF experiment at Fermilab, the Cosmic Microwave Background
astrophysics program at Berkeley Lab, the ATLAS experiment at CERN,
the DeepSearch supernova experiment, and the Relativistic Nuclear
Collisions program at Berkeley Lab.
In addition to the new servers and disk vaults, the expansion will
include added memory for some compute nodes as well as upgrades
to console servers, switches, networking cables, disk drives, software,
and miscellaneous tools -- all with the goal of continuing to provide
a reliable, well-supported resource that meets its users' growing