Differences

This shows you the differences between two versions of the page.

--- hpc:storage_on_hpc [2025/03/31 14:30] – [Sometimes moun is not available but you can browse/copy/interract with gio commands] Gaël Rossignol
+++ hpc:storage_on_hpc [2025/10/15 10:15] (current) – [NASAC] Adrien Albert
@@ Line 163: / Line 163: @@
 ====== Sharing files with other users ======
-Sometimes you need to share files with some colleagues or your research group.
+Sometimes, you may need to share files with colleagues or members of your research group.
-We provide two types of shared folders :
+We offer two types of shared folders:
-  * in "home" (''/home/share/'') - to share scripts and shared libraries for a common project
-  * in "scratch" (''/srv/beegfs/scratch/shares/'') - to share datasets for instance
-If you need one, please fill the form on DW: https://dw.unige.ch/openentry.html?tid=hpc
+  * **In the "home" directory** (`/home/share/`): Ideal for sharing scripts and common libraries related to a project.
+  * **In the "scratch" directory** (`/srv/beegfs/scratch/shares/`): Suitable for sharing larger files, such as datasets.
-If you are an Outisder user and you don't have access to DW,  please request to your PI to fill the form.
+To request a shared folder, please fill out the form at [[https://dw.unige.ch/openentry.html?tid=hpc|DW]]. As part of the request, you'll be asked if you already have a *group* you’d like to use. If this isn't the case, you'll need to create one ([[https://dw.unige.ch/openentry.html?tid=adaccess|link]] on the form)
+A **group** is a collection of users used to manage shared access to resources. These groups are defined and stored in the **Active Directory** and allow us to control who can access specific folders.
+If you need more details about groups, please contact your **CI** (*correspondant informatique*).
+If you are an *Outsider* user and do not have access to DW, please ask your **PI** to submit the request on your behalf.
 <note important>
 You are not allowed to change the permission of your ''$HOME/$SCRATCH'' folder on the clusters. Even if you did, our automation scripts will break what you did.
@@ Line 313: / Line 316: @@
 [sagon@login1 ~] $ dbus-launch bash
 </code>
+**If you are using sbatch add a sleep after ''dbus-launch'' to be sure initialisation is done**
+<code>
+dbus-launch bash
+sleep 3
+gio mount ....
+</code>¨
 mount the share, smb in this example:
@@ Line 384: / Line 395: @@
 <code>
-(bamboo)-[rossigng@login1 ~]$ dbus-launch bash
+$ dbus-launch bash
-(bamboo)-[rossigng@login1 ~]$ ll /run/user/
-/ 42/     451690/
+$ gio mount smb://nasac-evs2.unige.ch/hpc_exchange/backup
-(bamboo)-[rossigng@login1 ~]$ ll /run/user/
-total 0
-drwx------ 4 delislel unige     140 Jan 21 15:18 313457
-drwx------ 7 gdm      gdm       320 Jan 21 15:20 42
-drwx------ 4 stigler  hpc_users 140 Jan 21 15:18 451690
-(bamboo)-[rossigng@login1 ~]$ id
-uid=414558(rossigng) gid=5000(hpc_users) groups=5000(hpc_users),488(fuse),5001(hpc_admin),5044(hpc_allowed_users),5072(hpc_allowed_users_baobab),5073(hpc_allowed_users_yggdrasil),5074(hpc_allowed_users_bamboo)
-(bamboo)-[rossigng@login1 ~]$ gio mount smb://nasac-evs2.unige.ch/hpc_exchange/backup
 Authentication Required
 Enter user and password for share “hpc_exchange” on “nasac-evs2.unige.ch”:
@@ Line 400: / Line 403: @@
 Domain [SAMBA]: ISIS
 Password:
-(bamboo)-[rossigng@login1 ~]$ gio mount -l
+$ gio mount -l
 Drive(0): SAMSUNG MZ7L3480HBLT-00A07
   Type: GProxyDrive (GProxyVolumeMonitorUDisks2)
@@ Line 407: / Line 411: @@
 Mount(0): hpc_exchange on nasac-evs2.unige.ch -> smb://nasac-evs2.unige.ch/hpc_exchange/
   Type: GDaemonMount
-(bamboo)-[rossigng@login1 ~]$ ls /run/user/
-/ 42/     451690/
+$ gio list smb://nasac-evs2.unige.ch/hpc_exchange/
-(bamboo)-[rossigng@login1 ~]$ ls /run/user/414558
-ls: cannot access '/run/user/414558': No such file or directory
-(bamboo)-[rossigng@login1 ~]$ ls ~/.gvfs
-ls: cannot access '/home/users/r/rossigng/.gvfs': No such file or directory
-(bamboo)-[rossigng@login1 ~]$ ls .local/gvfs
-ls: cannot access '.local/gvfs': No such file or directory
-(bamboo)-[rossigng@login1 ~]$ ls .cache/gvfs
-(bamboo)-[rossigng@login1 ~]$ gio list smb://nasac-evs2.unige.ch/hpc_exchange/
 backup
-(bamboo)-[rossigng@login1 ~]$ gio list smb://nasac-evs2.unige.ch/hpc_exchange/backup
+$ gio list smb://nasac-evs2.unige.ch/hpc_exchange/backup
 toto
 titi
 tata.txt
+$ gio cp smb://nasac-evs2.unige.ch/hpc_exchange/backup/tata /tmp
+...
 </code>
@@ Line 457: / Line 458: @@
 The EESSI did a nice tutorial about CVMFS readable on [[https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/|multixscale]] git repo.
+===== EOS =====
+You can mount root filesystems using EOS.
+<code>
+(bamboo)-[sagon@login1 ~]$ export EOS_MGM_URL=root://eospublic.cern.ch
+(bamboo)-[sagon@login1 ~]$ export EOS_HOME=/eos/opendata
+(bamboo)-[sagon@login1 ~]$ eos fuse mount /tmp/sagon/opendata
+</code>
+<note important>do not mount the filesystem in your home or scratch space as it isn't working because they aren't standard filesystems</note>
 ====== Robinhood ======
 Robinhood Policy Engine is a versatile tool to manage contents of large file systems. It daily scans the scratch beegfs filesystems. It makes it possible to schedule mass action on filesystem entries by defining attribute-based policies.