Differences between revisions 32 and 33
|
Size: 5770
Comment:
|
Size: 5152
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 7: | Line 7: |
| * The users have at there disposal two types of filesystem: | * The users have at there disposal three types of filesystem: |
| Line 10: | Line 10: |
| * A '''CVMFS read-only filesystem''' for software provision | |
| Line 18: | Line 19: |
| * The fermi machines provide a wide set of filesystems so that users can access their data and applications. | |
| Line 20: | Line 20: |
| * The available filesystems in login machines are: | The '''pauli''' machines provide a wide set of filesystems so that users can access their data and applications. == NFS Home directories == Users home mounts are not shared with workernodes, they can be found only on submission nodes: |
| Line 23: | Line 27: |
| # df -h Filesystem Size Used Avail Use% Mounted on nfs01:/exports/exper-sw 2.8T 2.0T 767G 73% /exper-sw ---> NFS fs for local software se16:/ams 200G 33M 200G 1% /home/ams ---> NFS fs LIP homes and data se16:/comp 600G 6.8G 594G 2% /home/comp ---> NFS fs LIP homes and data se16:/cosmo 600G 8.8G 591G 2% /home/cosmo ---> NFS fs LIP homes and data se16:/feno 200G 33M 200G 1% /home/feno ---> NFS fs LIP homes and data se16:/nucria 600G 33M 600G 1% /home/nucria ---> NFS fs LIP homes and data se16:/pet 200G 33M 200G 1% /home/pet ---> NFS fs LIP homes and data se16:/sno 200G 8.2G 192G 5% /home/sno ---> NFS fs LIP homes and data se16:/t3atlas 200G 151G 49G 76% /home/t3atlas ---> NFS fs LIP homes and data se16:/t3cms 200G 29G 172G 15% /home/t3cms ---> NFS fs LIP homes and data se17:/x 4.6T 4.4T 165G 97% /x ---> NFS fs LIP ATLAS users se27:/ams 11T 5.9T 4.2T 59% /z/ams ---> NFS FS for AMS users se27:/comp 11T 3.8T 6.3T 38% /z/comp ---> NFS FS for COMP users se16:/csys 200G 33M 200G 1% /home/csys ---> Lustre FS for AUGER users mdt02@tcp:/t3atlas 61T 24T 34T 42% /gstore/t3atlas ---> Lustre FS Tier-3 LIP ATLAS users mdt02@tcp:/T3CMS 81T 11T 66T 15% /gstore/t3cms ---> Lustre FS Tier-3 LIP CMS users mdt03@tcp:/calo 72T 62T 6.6T 91% /lstore/calo ---> Lustre FS for ATLAS users mdt02@tcp:/cmslocal 6.4T 5.6T 541G 92% /lstore/cms ---> Lustre FS for CMS users mdt03@tcp:/comp 41T 9.0T 30T 24% /lstore/comp ---> Lustre FS for COMP users mdt03@tcp:/sno 10T 6.4T 3.2T 67% /lstore/sno ---> Lustre FS for SNOW users mdt04@tcp:/auger 36T 22T 12T 65% /lstore/auger ---> Lustre FS for AUGER users mdt04@tcp:/hpclip 6.4T 546M 6.0T 1% /lstore/hpclip ---> Lustre FS for HPC users mdt04@tcp:/lattes 5.9T 86G 5.5T 2% /lstore/lattes ---> Lustre FS for LATTS users mdt04@tcp:/pet 461G 115G 323G 27% /lstore/pet ---> Lustre FS for PET users cvmfs2 20G 17G 3.3G 84% /cvmfs/cms.cern.ch ---> CVMFS FS for CMS users cvmfs2 20G 6.1G 14G 32% /cvmfs/atlas.cern.ch ---> CVMFS FS for ATLAS users |
FILESYSTEM MOUNTED ON SCOPE se16:/ams /home/ams AMS se16:/comp /home/comp COMPASS se16:/cosmo /home/cosmo AUGER se16:/csys /home/csys CSYS se16:/feno /home/feno FENO se16:/nucria /home/nucria NUCRIA se16:/pet /home/pet PET se16:/sno /home/sno SNO PLUS se16:/t3atlas /home/t3atlas ATLAS se16:/t3cms /home/t3cms CMS |
| Line 53: | Line 40: |
| == NFS Data directories == Projects specific data NFS mounts are not shared with workernodes and are found only on submission nodes: {{{ se17:/x /x ATLAS data se27:/ams /z/ams AMS data se27:/comp /z/comp COMPASS data }}} == LUSTRE Data directories == Projects specific data LUSTRE mounts are shared with workernodes: {{{ mdt02@tcp:/t3atlas /gstore/t3atlas ATLAS Tier-3 mdt02@tcp:/T3CMS /gstore/t3cms CMS Tier-3 mdt04@tcp:/auger /lstore/auger AUGER mdt10@tcp:/atlasloc /lstore/atlas ATLAS Coimbra mdt04@tcp:/calo /lstore/calo ATLAS Lisboa mdt02@tcp:/cmslocal /lstore/cms CMS mdt03@tcp:/comp /lstore/comp COMP mdt04@tcp:/lattes /lstore/lattes LATTES mdt04@tcp:/nucria /lstore/nucria NUCRIA mdt04@tcp:/pet /lstore/pet PET mdt10@tcp:/sno /lstore/sno SNO PLUS }}} == CVMFS Software provision == The '''CVMFS''' is a read-only filesystem with cache local to clients optimal for software provision: {{{ cvmfs2 /cvmfs/cms.cern.ch CMS VO cvmfs2 /cvmfs/atlas.cern.ch ATLAS VO cvmfs2 /cvmfs/sw.el7 LIP }}} |
Contents
File systems available
- The users have at there disposal three types of filesystem:
A common NFS filesystem for homes not shared with workernodes
A lustre filesystem for some groups shared filesystem with the workernodes
A CVMFS read-only filesystem for software provision
- Groups involved in WLCG transfer data using grid technologies can also use the Tier-3 storage areas and dedicated software areas (CVMFS) made available by there communities.
Access Policy
- By default both lustre and nfs individual group areas can be accessed (read only) by users of the same group.
Directories and filesystems
The pauli machines provide a wide set of filesystems so that users can access their data and applications.
NFS Home directories
Users home mounts are not shared with workernodes, they can be found only on submission nodes:
FILESYSTEM MOUNTED ON SCOPE se16:/ams /home/ams AMS se16:/comp /home/comp COMPASS se16:/cosmo /home/cosmo AUGER se16:/csys /home/csys CSYS se16:/feno /home/feno FENO se16:/nucria /home/nucria NUCRIA se16:/pet /home/pet PET se16:/sno /home/sno SNO PLUS se16:/t3atlas /home/t3atlas ATLAS se16:/t3cms /home/t3cms CMS
NFS Data directories
Projects specific data NFS mounts are not shared with workernodes and are found only on submission nodes:
se17:/x /x ATLAS data se27:/ams /z/ams AMS data se27:/comp /z/comp COMPASS data
LUSTRE Data directories
Projects specific data LUSTRE mounts are shared with workernodes:
mdt02@tcp:/t3atlas /gstore/t3atlas ATLAS Tier-3 mdt02@tcp:/T3CMS /gstore/t3cms CMS Tier-3 mdt04@tcp:/auger /lstore/auger AUGER mdt10@tcp:/atlasloc /lstore/atlas ATLAS Coimbra mdt04@tcp:/calo /lstore/calo ATLAS Lisboa mdt02@tcp:/cmslocal /lstore/cms CMS mdt03@tcp:/comp /lstore/comp COMP mdt04@tcp:/lattes /lstore/lattes LATTES mdt04@tcp:/nucria /lstore/nucria NUCRIA mdt04@tcp:/pet /lstore/pet PET mdt10@tcp:/sno /lstore/sno SNO PLUS
CVMFS Software provision
The CVMFS is a read-only filesystem with cache local to clients optimal for software provision:
cvmfs2 /cvmfs/cms.cern.ch CMS VO cvmfs2 /cvmfs/atlas.cern.ch ATLAS VO cvmfs2 /cvmfs/sw.el7 LIP
Important Disclaimer
- Manipulating huge sets of small files generates performance degradation issues in /lustre due to the lock management. Therefore, you should not
Compile anything under /lustre
Store and access databases under /lustre
Access your data by SSHFS
Install sshfs package
On Ubuntu/debian:
# sudo apt-get install sshfs
On RHEL/CentOS/Fedora:
# sudo yum install fuse-sshfs
Create a local directory
# mkdir ~/homeNCG
Mount your remote folder
# sshfs martinsj@fermi.ncg.ingrid.pt: ~/homeNCG
When you're done, you can unmount it
# fusermount -u ~/homeNCG
.bashrc tunning
In case you're dealing with very long paths, as a suggestion, you can add to your ~/.bashrc file the following:
alias mfermi='f(){ dir=$1 ; [ -n "$dir" ] || dir=homeNCG ; [ -d ~/$dir ] || mkdir -p ~/$dir ; sshfs fermi.ncg.ingrid.pt: ~/$dir ; unset -f f; }; f'
alias ufermi='f(){ dir=$1 ; [ -n "$dir" ] || dir=homeNCG ; [ -d ~/$dir ] && fusermount -u ~/$dir ; [ -d ~/$dir ] && rmdir ~/$dir ; unset -f f; }; f'
alias mncg='f(){ dir=$1 ; [ -n "$dir" ] && ldir=`basename $dir` ; [ -d ~/$ldir ] || mkdir -p ~/$ldir ; [ -n "$dir$ldir" ] && sshfs fermi.ncg.ingrid.pt:$dir ~/$ldir ; unset -f f; }; f'
alias uncg='f(){ dir=$1 ; [ -n "$dir" ] && ldir=`basename $dir` ; ldir=`echo $ldir ; [ -d ~/$ldir ] && fusermount -u ~/$ldir ; [ -d ~/$ldir ] && rmdir ~/$ldir ; unset -f f; }; f'
alias mcalo='f(){ [ -d lcalo ] || mkdir ~/lcalo ; sshfs fermi.ncg.ingrid.pt:/lstore/calo ~/lcalo ; unset -f f; }; f'
alias ucalo='f(){ [ -d ~/lcalo ] && fusermount -u ~/lcalo ; [ -d ~/$ldir ] && rmdir ~/lcalo ; unset -f f; }; f'This is how you can use it:
$ mfermi homeNCG $ df Filesystem 1K-blocks Used Available Use% Mounted on fermi.ncg.ingrid.pt: 209612800 3504112 206108688 2% /home/martinsj/homeNCG $ ufermi homeNCG $ mncg /lstore/calo ... $ uncg /lstore/calo