File systems available

• The users have at there disposal two types of filesystem:

  • A common NFS filesystem for homes not shared with workernodes

  • A lustre filesystem for some groups shared filesystem with the workernodes

• Groups involved in WLCG transfer data using grid technologies can also use the Tier-3 storage areas and dedicated software areas (CVMFS) made available by there communities.

Access Policy

• By default both lustre and nfs individual group areas can be accessed (read only) by users of the same group.

Directories and filesystems

• The fermi machines provide a wide set of filesystems so that users can access their data and applications.
• The available filesystems in login machines are:

# df -h
Filesystem              Size  Used Avail Use% Mounted on
nfs01:/exports/exper-sw 2.8T  2.0T 767G  73% /exper-sw                  ---> NFS fs for local software
se16:/ams             200G   33M   200G   1% /home/ams                  ---> NFS fs LIP homes and data
se16:/comp            600G  6.8G   594G   2% /home/comp                 ---> NFS fs LIP homes and data
se16:/cosmo           600G  8.8G   591G   2% /home/cosmo                ---> NFS fs LIP homes and data
se16:/feno            200G   33M   200G   1% /home/feno                 ---> NFS fs LIP homes and data
se16:/nucria          600G   33M   600G   1% /home/nucria               ---> NFS fs LIP homes and data
se16:/pet             200G   33M   200G   1% /home/pet                  ---> NFS fs LIP homes and data
se16:/sno             200G  8.2G   192G   5% /home/sno                  ---> NFS fs LIP homes and data
se16:/t3atlas         200G  151G   49G   76% /home/t3atlas              ---> NFS fs LIP homes and data
se16:/t3cms           200G   29G   172G  15% /home/t3cms                ---> NFS fs LIP homes and data
se17:/x               4.6T  4.4T   165G  97% /x                         ---> NFS fs LIP ATLAS users
se27:/ams              11T  5.9T   4.2T  59% /z/ams                     ---> NFS    FS for AMS   users
se27:/comp             11T  3.8T   6.3T  38% /z/comp                    ---> NFS    FS for COMP  users
se16:/csys            200G   33M   200G   1% /home/csys                 ---> Lustre FS for AUGER users
mdt02@tcp:/t3atlas     61T   24T   34T   42% /gstore/t3atlas            ---> Lustre FS  Tier-3 LIP ATLAS users 
mdt02@tcp:/T3CMS       81T   11T   66T   15% /gstore/t3cms              ---> Lustre FS  Tier-3 LIP CMS   users
mdt03@tcp:/calo        72T   62T   6.6T  91% /lstore/calo               ---> Lustre FS for ATLAS users
mdt02@tcp:/cmslocal   6.4T  5.6T   541G  92% /lstore/cms                ---> Lustre FS for CMS   users
mdt03@tcp:/comp        41T  9.0T   30T   24% /lstore/comp               ---> Lustre FS for COMP  users
mdt03@tcp:/sno         10T  6.4T   3.2T  67% /lstore/sno                ---> Lustre FS for SNOW  users
mdt04@tcp:/auger       36T   22T   12T   65% /lstore/auger              ---> Lustre FS for AUGER users
mdt04@tcp:/hpclip     6.4T  546M   6.0T   1% /lstore/hpclip             ---> Lustre FS for HPC   users
mdt04@tcp:/lattes     5.9T   86G   5.5T   2% /lstore/lattes             ---> Lustre FS for LATTS users
mdt04@tcp:/pet        461G  115G   323G  27% /lstore/pet                ---> Lustre FS for PET   users
cvmfs2                 20G   17G   3.3G  84% /cvmfs/cms.cern.ch         ---> CVMFS  FS for CMS   users
cvmfs2                 20G  6.1G   14G   32% /cvmfs/atlas.cern.ch       ---> CVMFS  FS for ATLAS users

Important Disclaimer

• Manipulating huge sets of small files generates performance degradation issues in /lustre due to the lock management. Therefore, you should not

  • Compile anything under /lustre

  • Store and access databases under /lustre

Access your data by SSHFS

Install sshfs package

On Ubuntu/debian:

# sudo apt-get install sshfs

On RHEL/CentOS/Fedora:

# sudo yum install fuse-sshfs

Create a local directory

# mkdir ~/homeNCG

Mount your remote folder

# sshfs martinsj@fermi.ncg.ingrid.pt: ~/homeNCG

When you're done, you can unmount it

# fusermount -u ~/homeNCG

.bashrc tunning

In case you're dealing with very long paths, as a suggestion, you can add to your ~/.bashrc file the following:

alias mfermi='f(){ dir=$1 ; [ -n "$dir" ] || dir=homeNCG ; [ -d ~/$dir ] || mkdir -p ~/$dir ; sshfs fermi.ncg.ingrid.pt: ~/$dir ;  unset -f f; }; f'
alias ufermi='f(){ dir=$1 ; [ -n "$dir" ] || dir=homeNCG ; [ -d ~/$dir ] && fusermount -u ~/$dir ; [ -d ~/$dir ] && rmdir ~/$dir ;  unset -f f; }; f'

alias mncg='f(){ dir=$1 ; [ -n "$dir" ] && ldir=`basename $dir` ; [ -d ~/$ldir ] || mkdir -p ~/$ldir ; [ -n "$dir$ldir" ] && sshfs fermi.ncg.ingrid.pt:$dir ~/$ldir ;  unset -f f; }; f'
alias uncg='f(){ dir=$1 ; [ -n "$dir" ] && ldir=`basename $dir` ; ldir=`echo $ldir ; [ -d ~/$ldir ] && fusermount -u ~/$ldir ; [ -d ~/$ldir ] && rmdir ~/$ldir ;  unset -f f; }; f'

alias mcalo='f(){ [ -d lcalo ] || mkdir ~/lcalo ; sshfs fermi.ncg.ingrid.pt:/lstore/calo ~/lcalo ;  unset -f f; }; f'
alias ucalo='f(){ [ -d ~/lcalo ] && fusermount -u ~/lcalo ; [ -d ~/$ldir ] && rmdir ~/lcalo ;  unset -f f; }; f'

This is how you can use it:

$ mfermi homeNCG
$ df
Filesystem           1K-blocks    Used Available Use% Mounted on
fermi.ncg.ingrid.pt: 209612800 3504112 206108688   2%
/home/martinsj/homeNCG
$ ufermi homeNCG

$ mncg /lstore/calo
...
$ uncg /lstore/calo