User Tools

Site Tools


hpc:storage_on_hpc

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
hpc:storage_on_hpc [2025/04/07 10:02] Yann Sagonhpc:storage_on_hpc [2025/10/15 10:15] (current) – [NASAC] Adrien Albert
Line 170: Line 170:
   * **In the "scratch" directory** (`/srv/beegfs/scratch/shares/`): Suitable for sharing larger files, such as datasets.   * **In the "scratch" directory** (`/srv/beegfs/scratch/shares/`): Suitable for sharing larger files, such as datasets.
  
-To request a shared folder, please fill out the form at [[https://dw.unige.ch/openentry.html?tid=hpc|DW]]. As part of the request, you'll be asked if you already have a *group* you’d like to use.+To request a shared folder, please fill out the form at [[https://dw.unige.ch/openentry.html?tid=hpc|DW]]. As part of the request, you'll be asked if you already have a *group* you’d like to use. If this isn't the case, you'll need to create one ([[https://dw.unige.ch/openentry.html?tid=adaccess|link]] on the form)
  
 A **group** is a collection of users used to manage shared access to resources. These groups are defined and stored in the **Active Directory** and allow us to control who can access specific folders.   A **group** is a collection of users used to manage shared access to resources. These groups are defined and stored in the **Active Directory** and allow us to control who can access specific folders.  
Line 316: Line 316:
 [sagon@login1 ~] $ dbus-launch bash [sagon@login1 ~] $ dbus-launch bash
 </code> </code>
 +
 +**If you are using sbatch add a sleep after ''dbus-launch'' to be sure initialisation is done**
 +
 +<code>
 +dbus-launch bash
 +sleep 3
 +gio mount ....
 +</code>¨
  
 mount the share, smb in this example: mount the share, smb in this example:
Line 450: Line 458:
  
 The EESSI did a nice tutorial about CVMFS readable on [[https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/|multixscale]] git repo. The EESSI did a nice tutorial about CVMFS readable on [[https://multixscale.github.io/cvmfs-tutorial-hpc-best-practices/|multixscale]] git repo.
 +===== EOS =====
 +You can mount root filesystems using EOS.
  
 +<code>
 +(bamboo)-[sagon@login1 ~]$ export EOS_MGM_URL=root://eospublic.cern.ch
 +(bamboo)-[sagon@login1 ~]$ export EOS_HOME=/eos/opendata
 +(bamboo)-[sagon@login1 ~]$ eos fuse mount /tmp/sagon/opendata
 +</code>
  
 +<note important>do not mount the filesystem in your home or scratch space as it isn't working because they aren't standard filesystems</note>
 ====== Robinhood ====== ====== Robinhood ======
 Robinhood Policy Engine is a versatile tool to manage contents of large file systems. It daily scans the scratch beegfs filesystems. It makes it possible to schedule mass action on filesystem entries by defining attribute-based policies. Robinhood Policy Engine is a versatile tool to manage contents of large file systems. It daily scans the scratch beegfs filesystems. It makes it possible to schedule mass action on filesystem entries by defining attribute-based policies.
hpc/storage_on_hpc.1744020158.txt.gz · Last modified: (external edit)