{{METATOC 1-5}} ====== Access the clusters ====== ===== Account ===== You need an account on the HPC clusters to be able to access them and to submit jobs on them. To request an account please follow the procedure [[https://catalogue-si.unige.ch/en/hpc|on this page]] (Section: Related service). In principle Baobab/Yggdrasil are reserved for the members of the University of Geneva via their ISIs account (cf. https://catalogue-si.unige.ch/isis ) and to HES-SO GE users via a specific HPC account (cf. the procedure at https://catalogue-si.unige.ch/en/hpc ). But if you collaborate tightly with a researcher from another institution, the PI can provide him/her an access as an //independent contractor// (also called an //external collaborator// ) by following the next two steps: * Ask your [[https://memento.unige.ch/download_file/230/374|administrator]] to register the external member in the University database (cf. https://catalogue-si.unige.ch/en/isis ) * For more information about external accounts: https://memento.unige.ch/doc/0214/ * Proceed with the account request procedure as described in the preceding question. ==== Standard account ==== If you have a standard account (not external nor outsider), you can connect to the cluster and authenticate yourself by two mechanism: * you provide your ISIS user and password * you setup an ssh key in your [[https://my-account.unige.ch/|ISIS account]] and authenticate using your ssh key. ==== Outsider account ==== The users having an **outsider** account on the HPC cluster are requested to connect with the ssh key they provided during the invitation procedure described in the [[hpc:access_the_hpc_clusters#account|Account section]]. You can update your ssh key here by using this [[https://applicant.unige.ch/main/outsider-info/update|link]]. The UNIGE active directory synchronize itself with this application everyday at 1:30PM and 5:00AM. Your ssh key will be updated at these times. ===== Inactivity Notice and Account Deletion Policy ===== Accounts that have been inactive for a period of **one year** will be flagged for deletion. This is to ensure that we are not storing unused data and to avoid data being left without an owner in the event that the account is deleted from the UNIGE central user directory. Users will receive an email notification prior to the scheduled deletion of their account, giving them the opportunity to log in and keep their account active. If no response is received within one month, the account will be deleted, along with all associated data. ===== Cluster connection ===== Once you have received an email confirming the creation of you account, you have access to our clusters: Baobab and Yggdrasil. You can connect to the HPC clusters only through the login nodes. The clusters are reachable from outside UNIGE as well, without the need to use a VPN. ==== login nodes ==== * **For Baobab** : ''login1.baobab.hpc.unige.ch'' * **For Yggdrasil** : ''login1.yggdrasil.hpc.unige.ch'' * **For Bamboo** : ''login1.bamboo.hpc.unige.ch'' ==== Connect using SSH ==== You can access the clusters from anywhere through ''ssh'' with your [[https://catalogue-si.unige.ch/isis|ISIs]] account and password. === ssh PublicKey === If you want to authenticate to the cluster using your ssh key: 1. For security measure it isn't possible to add your ssh key to AuthorizedKeyFile. This has been disabled to prevent any non registered at UNIGE to log In. \\ 2. You ssh public key must be added to your UNIGE account profile. (it's like if the AuthorizedKeyFile is bonded to your UNIGE identity ) \\ 3. The UNIGE central directory synchronizes the ssh key every 10/15 min. \\ 4. More information on [[https://hpc-community.unige.ch/t/authentication-modification-sshpublickey-managment/3011]] **1. Generate your ssh-key** (We strongly recommend to create it with password/passphrase for more security) by following [[https://helpdeskgeek.com/how-to/how-to-generate-ssh-keys-on-windows-mac-and-linux/|this documentation (multi-platform)]] **2. Update your public** key to your Isis profile by updating "My SSH public key" on: * [[https://my-account.unige.ch|my-account]] for UNIGE/external users. * [[https://applicant.unige.ch/main/outsider-info/update|applicant]] for Outsider users. **Note**: Make sure you copy the public ssh key linked to the private key you're going to use. If you have regenerated your ssh key, you'll need to put your public key in [[https://my-account.unige.ch|my-account]] or [[https://applicant.unige.ch/main/outsider-info/update|applicant]] The first time you connect, your SSH client will ask you to confirm the **fingerprint** of the server. These are the current SSH fingerprint for both Baobab and Yggdrasil : [root@login1 ~]# ssh-keygen -l -E md5 -f /etc/ssh/ssh_host_rsa_key.pub 2048 MD5:8f:75:c4:18:8a:75:f1:f1:19:4d:85:92:3b:b6:2a:e1 no comment (RSA) Or this one (yggdrasil)-[root@login1 ~]$ ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key 2048 SHA256:tKqp4nljL+EGVKl8T0VF2nS36DkHVFMpLxQOPg/gKvg /etc/ssh/ssh_host_rsa_key.pub (RSA) If you connect using x2go (at least on windows), you'll see another **fingerprint**: [root@login1 ~]# for x in /etc/ssh/*.pub ; do echo $x && cut -d' ' -f2 < $x | base64 -d | openssl sha1 -c ; done ; /etc/ssh/ssh_host_rsa_key.pub (stdin)= 67:03:fc:6f:32:7c:19:9b:97:b9:e8:7b:12:1d:ad:a6:7b:c9:4c:9c If you type your username/password **wrong 3 times** in a row, you will be **banned for 15 minutes** before you can try again. Please read more in the [[hpc/access_the_hpc_clusters#troubleshooting|Troubleshooting]] section. === From Linux and Mac OS=== Connect to the login node from the terminal: ssh youruser@clusterhostname add ''-Y'' if you need X forwarding ssh -Y youruser@clusterhostname N.B. : replace ''clusterhostname'' with the [[hpc:access_the_hpc_clusters#login_nodes|login node]] of the cluster you are willing to use. === From Windows === To access Baobab or Yggdrasil in SSH from Windows, you can use : * ssh client integrated in PowerShell. Recent versions of Window 10 offer a ssh client. You can use it the same way you would use it on Linux (see [[hpc:access_the_hpc_clusters#from_linux_and_mac_os|above]]) * PuTTY which has been the //de facto// solution for years. * You can download it here [[http://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html|PuTTY]]. * You will find the needed information on the screenshots below. N.B. : replace ''clusterhostname'' with the [[hpc:access_the_hpc_clusters#login_nodes|login node]] of the cluster you are willing to use. PuTTY create session : {{ hpc:1-putty-create-session.png | PuTTY create session }} PuTTY enter username : {{ hpc:2-putty-enter-username.png | PuTTY enter username }} PuTTY save session : {{ hpc:3-putty-save-session.png | PuTTY save session }} PuTTY open session : {{ hpc:4-putty-open-session.png | PuTTY open session }} Once you open a session with PuTTY, you will be prompted for your password. | Password: Unlike Windows systems, Linux and Unix systems will **not display** **''*''** (star character) or any other character to indicate that you've entered something/anything in a password field - it simply stays totally blank as you type. Cursor won't blink, move, etc. \\ Just type your password and press enter, everything will be fine. ==== Access to the compute nodes ==== In case you need to access the compute nodes for purpose of debugging your software, using ''htop'' or other tool directly on the node, you need to go through the login node first and connect from there using ssh. ssh cpu001 It is important to note that you **cannot** access directly the compute nodes unless you have a **RUNNING job** on it. As soon as your job is finished, you will be logged out from the compute node. ===== GUI access / Desktop with X2Go ===== It's possible to access Baobab or Yggdrasil using a graphical interface. We support only ''X2Go''. You need to install [[http://wiki.x2go.org/doku.php/download:start|X2Go client]] on your computer and create a session following the screenshots below prior to connect to Baobab. X2Go create session : {{ hpc:1-x2go-client-session.png | X2Go create session }} Once connected to the Linux desktop, you can launch lightweight (image viewer, etc.) applications directly. If you need to launch a heavy graphical program such as Matlab for example, create a new interactive session (see the [[hpc:slurm#interactive_jobs|Interactive jobs]] section in the Slurm page) on a compute node. It is [[hpc:best_practices#rules_and_etiquette|forbidden]] to use the login node for this purpose. ===== File transfer ===== If you need to transfer files from your computer to Baobab or Yggdrasil, you need to use the ''sftp'' protocol (''scp''). ==== From Linux ==== We suggest you use ''scp'' or a program like ''rsync'' ==== From Windows ==== We suggest you to use [[https://filezilla-project.org/download.php?show_all=1|FileZilla client]] to transfer your files. do not download the sponsored version (the name sponsored appears in the download link), this installer may include bundled offers that may be recognized as virus Filezilla create session : {{ hpc:1-filezilla-create-session.png | Filezilla create session }} ===== SSH tunnel and socks proxy ===== If you want running JupyterLab or VScodeServer you may be interested by [[hpc:how_to_use_openondemand|OpenOnDemand]] The **login** nodes have a firewall that prevent incomming connection other than ssh. If you need to access a service from the cluster, please follow the: **1.** On your local machine, Save old ssh key and create a new one $ mkdir ~/.ssh/old $ mv ~/.ssh/* ~/.ssh/old $ ssh-keygen On the cluster, make sure you have not id_rsa key file (make a back up too) **2.** Copy the rsa.pub in [[https://my-account.unige.ch/main/home]] (for Unige Account) [[https://applicant.unige.ch/]] (for Outsider Account) and wait for 5-10 min the synchronisation with AD is done. the following command on login node should print your public ssh key registered in the AD: (baobab)-[alberta@login1 ~]$ /usr/bin/sss_ssh_authorizedkeys $USER ssh-rsa [...] **3.** On your local machine configure the proxyjump: [alberta@localhost ~]$ cat .ssh/config_baobab host baobab HostName login1.baobab.hpc.unige.ch User alberta Host cpu* HostName %h User alberta ProxyJump baobab Host gpu* HostName %h User alberta ProxyJump baobab [alberta@localhost ~]$ cat .ssh/config_yggdrasil host yggdrasil HostName login1.yggdrasil.hpc.unige.ch User alberta Host cpu* HostName %h User alberta ProxyJump yggdrasil Host gpu* HostName %h User alberta ProxyJump yggdrasil **4.** Alloc a test job and open a new tab on your local machine and try to connect on the allocated node: **On baobab:** (baobab)-[alberta@login1 ~]$ salloc --time=00:05:00 salloc: Pending job allocation 5574654 salloc: job 5574654 queued and waiting for resources salloc: job 5574654 has been allocated resources salloc: Granted job allocation 5574654 salloc: Waiting for resource configuration salloc: Nodes cpu001 are ready for job At the same time On your local machine, connect to the compute with selecting the right ssh config file (For this example: Baobab): ( My first test was on cpu026 this is the message a i got) [alberta@localhost .ssh]$ ssh -F .ssh/config_baobab cpu026 The authenticity of host 'cpu026 ()' can't be established. RSA key fingerprint is SHA256:tKqp4nljL+EGVKl8T0VF2nS36DkHVFMpLxQOPg/gKvg. RSA key fingerprint is MD5:8f:75:c4:18:8a:75:f1:f1:19:4d:85:92:3b:b6:2a:e1. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'cpu026' (RSA) to the list of known hosts. Last login: Tue Oct 24 10:49:29 2023 Installed: Thu Aug 17 14:40:08 CEST 2023 But working on cpu001 as well: [alberta@localhost ~]$ ssh -F .ssh/config_baobab cpu001 Last login: Mon Oct 23 16:43:34 2023 Installed: Thu Aug 17 14:28:26 CEST 2023 (baobab)-[alberta@cpu001 ~]$ More Information on HPC-community forum: [[https://hpc-community.unige.ch/t/tutorial-ssh-tunneling-and-socks-proxy/1795]] [[https://hpc-community.unige.ch/t/proxyjump-ssh-not-working-on-baobab/3078/15]]