Documentation utilisateur

NB, part of the following instructions come from ¹⁾.

NB, this originates from ²⁾.

First of all, which snapshots do we keep?

• 4hourly : 1 snapshot chaque 4 heures, avec une queue de 42. Nous conservons donc un historique d'une semaine.

• weekly : 1 snapshot par semaine (le dimanche) avec une queue de 6. Nous conservons donc un historique d'un mois et demi (6 semaines).

Then, how to access them?

• Windows: in the File Explorer:
  ⇒ right-click on the folder (either at the share root or at any sub-folder), then Properties → Previous Versions
• Mac or Linux:
  ⇒ enter the .snapshot hidden folder

NB, this originates from ³⁾.

To assure the service continuity in case of problems in the primary UNIGE DC at UniDufour, a nightly replication is automatically done at the secondary UNIGE DC at the Campus Biotech.

The main differences for the NAS infrastructure at the Campus Biotech are:

1. 2 old nodes (4080 models vs. the 5300 at UniDufour)
2. nor 4hourly neither weekly snapshots, but daily ones (automatically taken before the nightly replication starts)
3. no end-user access

NB, this originates from ⁴⁾.

For anti-ransomware reasons, we also have a backup on tape of each NAS share, thus completely independent from the NAS infrastructure.

Some notes:

1. while our goal is to have 3day backups, we assure only at least a weekly backup.
2. data that has been deleted from the NAS share is kept up to 2 years after the deletion date.

Then, how to access them?

NB, this originates from ^{5) 6)}

If you want to migrated NAS data (actually, any data…) to the UNIGE tape solution (cf. https://catalogue-si.unige.ch/tape, based on IBM Spectrum Archive), you can use the ug-nas-create-zip-archive-from-folder.sh from the ug-nas project, which:

1. calculate the size of the source folder
2. calculate the MD5 hashes for each file in the source folder
3. create a 2GB-by-default split (AKA multiple-file) .zip archive in the destination folder
4. list the destination folder content
5. delete the source folder and all its content

$ ug-nas-create-zip-archive-from-folder.sh -h
UG-NAS: Create a split ZIP archive from a NAS folder
Usage: ug-nas-create-zip-archive-from-folder.sh [-h] [-s <splitsize_gb>] <source> <target>

Options:
     -h : display this help and exit
     -s : split at GB (defaults to 2)

$ ug-nas-create-zip-archive-from-folder.sh \
    "/path/to/source/folder" \
    "/path/to/destination/folder" \
    2>&1 | \
 tee "/path/to/log/folder/$(date +%Y%d%m)_ug-nas-create-zip-archive-from-folder_${SOURCE_FOLDER}.log"
Thu 16 Nov 2023 11:48:49 AM CET
=====
I: calculating source folder size...
3.9G
=====
Thu 16 Nov 2023 11:48:49 AM CET
=====
I: generating MD5 checksum for each file in source folder...
[...]

real    1m3.094s
user    0m7.158s
sys     0m1.736s
=====
Thu 16 Nov 2023 11:49:52 AM CET
=====
I: creating the split ZIP archive...
splitsize = 2147483648
[...]
total bytes=4098480958, compressed=3899876027 -> 5% savings

real    2m48.380s
user    2m30.320s
sys     0m5.719s
=====
Thu 16 Nov 2023 11:52:40 AM CET
=====
I: listing archive folder '/path/to/destination/folder' content:
total 3768321
[...]
=====
Thu 16 Nov 2023 11:52:40 AM CET
=====
I: deleting the source folder...

real    0m0.073s
user    0m0.004s
sys     0m0.029s
=====
Thu 16 Nov 2023 11:52:40 AM CET
=====
I: all done!
$ 

NB, this originates from ⁷⁾.

The UNIGE central NAS is available only from within the UNIGE internal network (including the VPN access).

Depending on the access type you need from the outside world, you should:

1. ask to activate the share in your EduCloud account (cf. https://plone.unige.ch/distic/pub/logiciels/filr-switchdrive) if access is restricted on specific files
2. ask for an external UNIGE ISIs account (cf. https://catalogue-si.unige.ch/en/isis) if access to a large amount of data is needed or analysis must be directly made on such data (e.g. for a independent contractor)
3. duplicate the data stored on the UNIGE central NAS to the UNIGE central S3 service and then create pre-signed S3 URLs (cf. https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html)
   ATTENTION, this means you must pay both the NAS and the S3 spaces!

NB, the following instructions come from ^{8) 9)}

If you need to synchronize data to another folder, you can let `rsync` 3.1.0+ saves in the log file the MD5 checksum of any transferred file (cf. https://stackoverflow.com/questions/29624524/how-can-i-print-log-the-checksum-calculated-by-rsync#45053057):

$ rsync \
      --log-file=/path/to/file.log \
      --log-file-format="%C  %f" \
      [...] \
      "${SOURCE_FOLDER}" \
      "${TARGET_FOLDER}"
$ grep -e "${SOURCE_FOLDER}" /path/to/file.log | \
      cut -c 29- | \
      awk '{if ($2 != "") {print $0}}' | \
      sed -e "s, ${SOURCE_FOLDER}/, ,g" \
      >"/path/to/${SOURCE_FOLDER}.md5"
$ cd "${TARGET_FOLDER}"
$ md5sum -c "/path/to/${SOURCE_FOLDER}.md5"

TreeSize est lancé sur certains share de manière périodique.

https://plone.unige.ch/distic/acteurs-du-si/distic/prods/pole-environnement-de-travail-et-integration/serveurs/serveur-de-statistique/liste-des-partages-scanner

NB, this partly originates from ^{10) 11) 12)}.

There are 3 different actors involved in the life of an SMB share:

1. the DiSTIC RISE storage team
   ⇒ the infrastructure administrators and share configurators (allocated space, ACL administrative access and pass-through access)
2. the share owner (AKA contact_owner)
   ⇒ the one responsible for the share content (thus deciding who has access to them), also providing the billing information
3. the share administrators (AKA contact_admin)
   ⇒ the share content managers

Please follow the links on the catalogue des services informatiques, section "Obtenir une prestation" to ask for a new share or a modification to an existing one, as well as to report a problem.

Lorsqu'un partage a été crée et que votre CI vous a donné les droits d'accès (cf. how to manage the ACLs (AKA permissions)), suivez la procédure ci-dessous pour vous connecter à votre partage. A noter que pour vous connecter à un partage depuis l'extérieur de l'université, vous devez configurer et lancer votre VPN. Veuillez vous référer à la documentation du VPN de l'unige pour ce faire.

NB, this originates from ^{13) 14)}

Les produits Adobe sont de plus en plus “strictes” par rapport aux disques réseaux :

1. Photoshop ne support pas le disques réseaux :
   https://support-si.unige.ch/openentry.html?tid=INC000000281029
   https://helpx.adobe.com/photoshop/kb/networks-removable-media-photoshop.html
2. Acrobat “Protected Mode” ne permet pas de sauvegarder des PDFs sur des disques réseaux :
   https://community.adobe.com/t5/acrobat-reader-discussions/save-as-in-network-drive-not-possible/td-p/9000860?profile.language=fr
   https://community.adobe.com/t5/acrobat-discussions/error-while-saving-a-pdf-file-to-a-mapped-drive/td-p/5270686

First of all, you need to mount the share root (thus, not a super/sub-folder).

Then

• Windows: in the File Explorer:
  ⇒ right-click on the folder (either at the share root or at any sub-folder), then General → Capacity
• Mac or Linux:
  ⇒ in a terminal

  df -h /path/to/the/mounted/share

NB, part of the following instructions come from ¹⁵⁾.

Here the commands to get the list of all the shares you have access on a specific server:

1. Windows ¹⁶⁾:

   PS C:\WINDOWS\system32> net view \\nasac-faculty.unige.ch /all | 
    Select-String Disk
   [...]
   PS C:\WINDOWS\system32> (net view \\nasac-faculty.unige.ch /all | 
    Select-String Disk).length
   107
   PS C:\WINDOWS\system32> 

2. GNU/Linux:
   ATTENTION, unfortunately 'smbclient -L' does not allow “grep-piping” if interactive!

   capello@harlock:~$ smbclient -L nasac-faculty.unige.ch -W ISIS -U capello
   Enter ISIS\capello's password: 
    
   	Sharename       Type      Comment
   	---------       ----      -------
   [...]
   SMB1 disabled -- no workgroup available
   capello@harlock:~$ cat <<EOF >~/.smbclient
   username = capello
   password = ${ISIS}
   domain   = ISIs
   EOF
   capello@harlock:~$ smbclient -L nasac-faculty.unige.ch -W ISIS -A ~/.smbclient | grep -c Disk
   107
   capello@harlock:~$ 

La procédure est sensiblement la même pour d'autres variantes de Linux.

NB, this originates from ¹⁷⁾.

1. via plain old mount command, which however requires superuser privileges and the cifs kernel module:

   root@harlock:~# findmnt /mnt
   root@harlock:~# mount \
       -t cifs \
       -o vers=3,sec=ntlmsspi,domainauto,username=capello \
       //nasac-faculty.isis.unige.ch/ADM_HOME/DISTIC/capello \
       /mnt/
   Password for capello@//nasac-faculty.isis.unige.ch/ADM_HOME/DISTIC/capello: 
   root@harlock:~# findmnt /mnt 
   TARGET SOURCE                                                FSTYPE OPTIONS
   /mnt   //nasac-faculty.isis.unige.ch/ADM_HOME/DISTIC/capello cifs   rw,relatime,vers=3,sec=ntlmsspi,cache=strict,username=capello,domain=ISIS,uid=0,noforceuid,gid=0,noforcegi
   root@harlock:~# 

2. via GVfs/Gio (i.e. the same “helper” framework used by most of the graphical tools including the default Ubuntu desktop), which is an unprivileged action with its own non-kernel SMB implementation:

   capello@harlock:~$ gio mount smb://nasac-faculty.isis.unige.ch/ADM_HOME/DISTIC/capello
   Password required for share adm_home on nasac-faculty.isis.unige.ch
   User [capello]: 
   Domain [WORKGROUP]: ISIs
   Password: 
   capello@harlock:~$ ls -l /run/user/$(id -u)/gvfs/
   total 2
   drwx------ 1 capello capello 2048 Nov 30 11:50 smb-share:server=nasac-faculty.isis.unige.ch,share=adm_home
   capello@harlock:~$ 

Se connecter à un serveur:

Choisir l'ip ou le nom de votre EVS (transmis par votre CI):

Entrer vos identifiants ISIS:

NB , the following instructions come from <https://gitlab.unige.ch/prods/ies/recherche/hpc/issues/925>

SMB3+ requires packet signing, thus if you encounter the following error…

CIFS VFS: validate protocol negotiate failed: -13
CIFS VFS: failed to connect to IPC (rc=-5)
CIFS VFS: validate protocol negotiate failed: -13
CIFS VFS: session ffff962f7cd42400 has no tcon available for a dfs referral request
CIFS VFS: cifs_mount failed w/return code = -5

…you have to specify that you want packet signing via a mount option, either sec=ntlmsspi (preferred) or sec=ntlmv2.

NB , the following instructions come from <https://gitlab.unige.ch/prods/ies/recherche/hpc/issues/873>

Starting from Windows 10, symlinks (AKA symbolic links) are fully supported (cf. https://blogs.windows.com/windowsdeveloper/2016/12/02/symlinks-windows-10/ ). However, while Linux can make use of symlinks on a SMB share created on Windows 10, there is no way to create them on Linux.

On the other hand, Linux-only symlinks on SMB shares are still possible, no more via the SMB1-only CIFS Unix Extensions (cf. https://www.samba.org/samba/CIFS_POSIX_extensions.html ), but via the new SMB3 POSIX Extensions (cf. https://wiki.samba.org/index.php/SMB3-Linux#Symbolic_links ).

Specyfing the mount option mfsymlinks will allow symlink creation on Linux using the Minshall+French file format, which means that they will be seen on Windows as a plain text file (cf. https://wiki.samba.org/index.php/UNIX_Extensions#Storing_symlinks_on_Windows_servers ).

ATTENTION , while the mount option mfsymlinks is available from SMB2+, it is preferable to use it together with SMB3+ (see Linux and SMB3+).

Connectez un lecteur réseau pour y accéder à partir de l’Explorateur de fichiers dans Windows sans avoir à le rechercher ou à saisir son adresse réseau à chaque fois.

1. Ouvrez l'Explorateur de fichiers à partir de la barre des tâches ou du menu Démarrer, ou appuyez sur la Touche de logo Windows + e.
   
2. Sélectionnez Ce PC dans le volet de gauche. Puis, dans l'onglet Ordinateur, sélectionnez Connecter un lecteur réseau

Sélection du disque et du nom du partage:

Saisie de vos identifiants ISIS:

NB, this originates from ^{18) 19) 20)}

If you still get connection refused messages and you are sure your ISIs credentials are right, the IP address your computer got could have been blacklisted in the last 15 minutes on the NAS side because of too many SMB NTLM authentication failures.

Here how to find out if you are impacted:

1. find out your current IP address:
   1. Windows: Start → PowerShell → ipconfig → IPv4 Address
   2. Mac: Applications → Terminal → ifconfig | grep -e “inet ”
   3. Linux: Applications → Terminal → ip addr show | grep -e “inet ”
2. check if your IP is listed in the last-unblocked-IP on https://luninasadm1.unige.ch/nas-smb-barred-clients-list.txt

NB, part of the following instructions come from ^{21) 22) 23)}.

You need at least 2 AD groups for an SMB share to be created:

1. an admin one to manage the ACLs (AKA permissions)
   ⇒ full control at the first level (thus \\${SERVER}\${SHARE}\), with inheritance enabled
2. a pass-through one to be able to mount/connect the share
   ⇒ list the first level content only (thus no inheritance)

These 2 groups will then be used to set the ACLs of the share itself (thus the mount point), ACLs that must be modified via the administrative interface only.

Once the share has been created, you must set the ACLs at least for the 1st-level content, directly from Windows:

• if you are a Correspondant Informatique, please check the internal documentation “Affectation des droits” paragraph in the “Gestion des postes de travail avec l'Active Directory (AD)” guide available on Plone.
• if you are a end-user, please check the external documentation at https://wiki.samba.org/index.php/Setting_up_a_Share_Using_Windows_ACLs#Setting_ACLs_on_a_Folder or https://docs.netapp.com/us-en/ontap/smb-admin/configure-ntfs-windows-security-tab-task.html .

NB , the following instructions come from ^{24) 25)}.

Nowadays most of the GNU/Linux distribution defaults to NFSv4, which is a big step forward and differs quite a lot from NFSv3 (cf. http://www.citi.umich.edu/projects/nfsv4/ ). One of the main advantages of NFSv4 is the ACLs support into the protocol itself (cf. http://wiki.linux-nfs.org/wiki/index.php/ACLs ), ACLs that resemble very much the Windows ones.

The NASAC, being based on a Unix system, natively supports NFSv4 ACLs. Actually, given that it is connected to the UniGE's Active Directory, the NFSv4 ACLs are the same as the Windows ones.

Here the instructions to manage the NFSv4 ACLs for a share:

1. client mapping does not matter, knowing that when no local mapping is possible everything is nobody:nobody (cf. https://www.thegeekdiary.com/nfsv4-mountpoint-shows-incorrect-ownerships-as-nobodynobody-in-centos-rhel/ ).
   ATTENTION :
   • on RHEL 6.3+ NFS ID mapping is disabled by default, _i.e._ /sys/module/nfs/parameters/nfs4_disable_idmapping is set to Y (cf. https://access.redhat.com/articles/2252881 ), thus numeric UIDs/GIDs are sent over the wire.
   • NFSv4 ID mapping on clients is now managed by a separate process, nfsidmap (which does not run as a daemon), thus you no more need rpm.idmapd . Nevertheless, they share the same configuration file, _i.e._ /etc/idmapd.conf .
2. server mapping is mandatory, thus ask the NASAC administrator to map the needed users/groups via support-si .
3. please DO NOT USE chmod to set any POSIX permissions, given that this actually interacts with the NFS4 ACLs, setting the latter to reflect the POSIX permissions just set (cf. http://wiki.linux-nfs.org/wiki/index.php/ACLs#Server and http://wiki.linux-nfs.org/wiki/index.php/ACLs#Strict_Mapping ).
4. it is thus not possible to combine both, POSIX permissions and NFS4 ACLs, but you can combine POSIX ownership and NFSv4 ACLs.
5. the NFSv4 ACLs order matters and try to avoid DENY ACLs, they are evaluated before ALLOW ones (cf. https://docs.microsoft.com/en-us/windows/win32/secauthz/dacls-and-aces and https://docs.microsoft.com/en-us/windows/win32/secauthz/order-of-aces-in-a-dacl ).
6. special care must be taken if the NFSv4 share is also available via CIFS, given that the NFSv4 ACLs are also the Windows ones and thus errors done on the CIFS share have an impact on the NFSv4 share as well.
7. posixuser:root is now able to access the NFSv4 share in read/write mode without any server modification (_i.e._ the NFSv3 rootsquash option is no more available).
8. the nfs4-acl-tools package (both .deb and .rpm) provides the nfs4_[edit|get|set]facl programs to manage the NFSv4 ACLs (cf. their manpages, man 5 nfs4_acl or https://www.osc.edu/book/export/html/4523 for an overview).