Differences

This shows you the differences between two versions of the page.

--- hpc:applications_and_libraries [2023/10/13 15:31]
Adrien Albert [R packages]
+++ hpc:applications_and_libraries [2024/03/25 16:52] (current)
Adrien Albert [Pull an existing image]
@@ Line 217: / Line 217: @@
 |foss/2022b |gcc 12.2.0 |openmpi 4.1.4     |
 |foss/2023a |gcc 12.3.0 |openmpi 4.1.5     |
+|foss/2023b |gcc 13.2.0 |openmpi 4.1.6     |
@@ Line 429: / Line 430: @@
 capello@login2:~$
 </code>
 ===== Jupyter notebook and Jupyter lab=====
@@ Line 836: / Line 838: @@
 ~/baobab_python_env/bin/python
 </code>
+==== Pip install from source ====
+By default when you use ''pip'' to install a library, it will download a binary of ''.whl'' file instead of building the module from source. This may be an issue if the module itself depend on a custom ''libc'' version or is optimized for a given kind of CPU. In this case, you can force ''pip'' to install a module by building it from source.
+Example to build hpy from source. The ''h5py'' argument to ''--no-binary'' is to specify that you want to build from source only ''h5py''.
+<code>
+pip install --no-binary h5py h5py
+</code>
 ===== R project and RStudio =====
@@ Line 997: / Line 1008: @@
 ===== Variant Effect Predictor (VEP) =====
+This tutorials is inspired from VEP documentation: [[https://www.ensembl.org/info/docs/tools/vep/script/vep_download.html#singularity]]
 According vep maintainers (https://github.com/Ensembl/ensembl-vep/issues/1515) each users should have one instance of vep. Please edit destination use the following sbatch to install your instance.
@@ Line 1024: / Line 1037: @@
 srun mkdir vep_data
 srun singularity exec vep.sif INSTALL.pl -c vep_data -a p -g all -r vep_data
-srun mkdir -p $HOME/vep_singularity/install_dir_shm
+srun mkdir -p $HOME/vep_singularity/install_dir
-srun mv * $HOME/vep_singularity/install_dir_shm
+srun mv * $HOME/vep_singularity/install_dir
 </code>
-To download species after the installation run (it assumes you have the same directory arch as above):
+==== Install species  ====
-//It ran the command install on debug-cpu assuming the install required less than 15 min otherwise use another partition with --time=HH:mm:ss//
+To download species after the installation run the following cmd line(it assumes you have the same directory arch as above):\\
+//It ran the  install command on debug-cpu slurm partition assuming the install required less than 15 min otherwise use another partition with --time=HH:mm:ss//
-  srun singularity exec vep.sif INSTALL.pl -c vep_data -a cf -s <species_name>
+  (baobab)-[alberta@login2 ~]$ cd /home/users/a/alberta/vep_singularity/install_dir
+  (baobab)-[alberta@login2 ~]$ srun singularity exec vep.sif INSTALL.pl -c vep_data -a cf -s <species_name>
@@ Line 1040: / Line 1054: @@
 ==== Intro ====
-Apptainer is a Docker like for HPC. It is available directly on the OS, not through module.
+Apptainer is a Docker like for HPC. It is available directly on the OS, not through module. \\
+As you don't have root access on Baobab, you cannot build recipe file on Baobab. If you need to do this, you may want to build the image on your own machine and transfer the image to Baobab.\\
+You can download existing images from shub (singularity hub) or from docker (docker public or private registry).\\
+The image will be converted to a read-only squashfs on Baobab disk. It's not possible to have writable images as user. Instead you should build the image as sandbox.\\
-As you don't have root access on Baobab, you cannot build recipe file on Baobab. If you need to do this, you may want to build the image on your own machine and transfer the image to Baobab.
+Apptainer images are immutable, but it is possible to append an overlay, see below.
-You can download existing images from shub (singularity hub) or from docker (docker public or private registry).
+=== Pull an existing image ===
+<WRAP round alert 50%>
+ Do not run the following commands on login node !! It's using too much cpu
+</WRAP>
+Exemple with Rstudio:
-The image will be converted to a read-only squashfs on Baobab disk. It's not possible to have writable images as user. Instead you should build the image as sandbox.
+**Create a project directory and pull the image from compute node:**
+  (baobab)-[alberta@login2 ~]$ MYPROJECT="rocker"
-Apptainer images are immutable, but it is possible to append an overlay, see below.
+  (baobab)-[alberta@login2 ~]$ mkdir -p singularity/$MYPROJECT
+  (baobab)-[alberta@login2 ~]$ cd !$
+  (baobab)-[alberta@login2 rocker]$ salloc --partition=shared-cpu --time=00:30:00 --cpus-per-task 12
+  (baobab)-[alberta@cpu300 rocker]$ apptainer pull docker://rocker/rstudio:4.2
+  (baobab)-[alberta@cpu300 rocker]$ ls
+  rstudio_4.2.sif
+And Voila; I get my sif image rstudio_4.2.sif
 ==== Convert a Docker image ====

eResearch Doc

User Tools

Site Tools

Differences

Page Tools