With great success at SuperComputing 2006 (SC06), held November 11-17 in Tampa Bay, Florida, USA, a team from ACCRE demonstrated the latest 0.8 version of Logistical Storage software (L-Store), our wide area network storage solution. L-Store has been indigenously developed within ACCRE to address the storage needs of researchers at Vanderbilt University. L-Store was first demonstrated at SuperComputing 2005 in Seattle, Washington, USA. The system sustained writes in excess of 10 Gigabits/s for extended periods of time. A typical run wrote 1.2 TB of data in approximately 17 minutes. The ACCRE team made significant improvements over the next year. At SC06, using two racks of equipment connected by four fibre network runs, the L-Store system was able to achieve sustained data transfers at 3.3 GigaBytes/s. Given each fibre can handle 1 GigaByte/s, the system came very close to utilizing the available bandwidth, i. e., "filling the pipe". This rate is equivalent to writing over 5 CDROMs per second, and we could have filled 100 TB of disk space in under 8 hours. This high-volume traffic generated by our system is typical of the kind that will be commonplace in many research areas in the near future, and is already strongly in demand for many fields including the High Energy Physics and Molecular Physiology communities. Below is a plot of a typical run lasting two hours.

Disk I/O Statistics per Depot: Each color band represents a single depot. Each Capricorn box has 4 depots.

In addition to the local network, the ACCRE team also had access through collaborators to systems at Caltech in Pasadena, California and was able to participate in the bandwidth challenge. The system sustained 10 Gigabits/s over the 1 GigaByte/s National Lambda Rail backbone wide area network (WAN) between Pasadena and Tampa Bay with minimal hardware; we had roughly 100 TB on the floor. As the following graphic demonstrates we effectively utilized the network, impressing the network engineers working with us from Caltech. Note that at the beginning of the run we were only writing, while near the end we were writing and reading, using even more bandwidth when doing both. See also the press release at PhysOrg.

Network Statistics per Box: Each color band represents a single Capricorn box. Each Capricorn box has 4 depots.

These feats were accomplished using commodity hardware from Capricorn Technologies, the Foundry Networks SuperX FastIron SX800 switch, and assistance from Serious Systems. The cost of the storage was inexpensive. L-Store can be built using many types of commodity hardware. Again, this achievement was significant given such capabilities will become essential for many areas of research.

This work is supported funding from NSF grant PHY-0619847 and by the Vanderbilt Center for the Americas .