Skip to content

Stash service and CephFS cluster

Introduction

The purpose of the stash.osgconnect.net server is to provide access to data storage for distributed computing projects, like osgconnect projects. The stash.osgconnect.net server provides access to a CephFS cluster through the web and other software.

Hardware on the CephFS cluster

The CephFS cluster has 33 nodes from three generations of purchases.

Generation 1 nodes

Quantity14
ModelDell PowerEdge R730xd
Processor2x Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
Memory6x 16GB DDR4, 96 GB total
Disk2x 1TB 7.2k RPM HDD SATA for system, 12x 6TB 7.2k RPM HDD SAS for Ceph

Generation 2 nodes

Quantity17
ModelDell PowerEdge R730xd
Processor2x Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
Memory6x 16GB DDR4, 96 GB total
Disk2x 1TB 7.2k RPM HDD SATA for system, 16x 8TB 7.2k RPM HDD SAS for Ceph

Generation 3 nodes

Quantity2
ModelDell PowerEdge R740xd
Processor2x Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz
Memory12x 16GB DDR4, 192 GB total
Disk2x 240GB BOSS for system, 18x 12TB 7.2k RPM HDD SAS for Ceph

Software on the stash server

  • nginx server
  • xrootd server

Web access via stash

xrootd access via stash

You can access the Ceph cluster with xrootd. Below is an example of copying a file with xrootd.

xrdcp root://stash.osgconnect.net:1094//osgconnect/public/<user_id>/<file> -> /tmp

stashcp file_name stash://stash.osgconnect.net/

stashcp tool for copying data

You can copy files to the Ceph cluster using the stashcp tool.

stashcp output.dat stash:///osgconnect/public/<username>/output/output.dat