Differences

This shows you the differences between two versions of the page.

--- hpc:applications_and_libraries [2023/10/13 17:04]
Adrien Albert [Variant Effect Predictor (VEP)]
+++ hpc:applications_and_libraries [2024/03/25 16:52] (current)
Adrien Albert [Pull an existing image]
@@ Line 217: / Line 217: @@
 |foss/2022b |gcc 12.2.0 |openmpi 4.1.4     |
 |foss/2023a |gcc 12.3.0 |openmpi 4.1.5     |
+|foss/2023b |gcc 13.2.0 |openmpi 4.1.6     |
@@ Line 429: / Line 430: @@
 capello@login2:~$
 </code>
 ===== Jupyter notebook and Jupyter lab=====
@@ Line 836: / Line 838: @@
 ~/baobab_python_env/bin/python
 </code>
+==== Pip install from source ====
+By default when you use ''pip'' to install a library, it will download a binary of ''.whl'' file instead of building the module from source. This may be an issue if the module itself depend on a custom ''libc'' version or is optimized for a given kind of CPU. In this case, you can force ''pip'' to install a module by building it from source.
+Example to build hpy from source. The ''h5py'' argument to ''--no-binary'' is to specify that you want to build from source only ''h5py''.
+<code>
+pip install --no-binary h5py h5py
+</code>
 ===== R project and RStudio =====
@@ Line 1043: / Line 1054: @@
 ==== Intro ====
-Apptainer is a Docker like for HPC. It is available directly on the OS, not through module.
+Apptainer is a Docker like for HPC. It is available directly on the OS, not through module. \\
+As you don't have root access on Baobab, you cannot build recipe file on Baobab. If you need to do this, you may want to build the image on your own machine and transfer the image to Baobab.\\
+You can download existing images from shub (singularity hub) or from docker (docker public or private registry).\\
+The image will be converted to a read-only squashfs on Baobab disk. It's not possible to have writable images as user. Instead you should build the image as sandbox.\\
-As you don't have root access on Baobab, you cannot build recipe file on Baobab. If you need to do this, you may want to build the image on your own machine and transfer the image to Baobab.
+Apptainer images are immutable, but it is possible to append an overlay, see below.
-You can download existing images from shub (singularity hub) or from docker (docker public or private registry).
+=== Pull an existing image ===
+<WRAP round alert 50%>
+ Do not run the following commands on login node !! It's using too much cpu
+</WRAP>
+Exemple with Rstudio:
-The image will be converted to a read-only squashfs on Baobab disk. It's not possible to have writable images as user. Instead you should build the image as sandbox.
+**Create a project directory and pull the image from compute node:**
+  (baobab)-[alberta@login2 ~]$ MYPROJECT="rocker"
-Apptainer images are immutable, but it is possible to append an overlay, see below.
+  (baobab)-[alberta@login2 ~]$ mkdir -p singularity/$MYPROJECT
+  (baobab)-[alberta@login2 ~]$ cd !$
+  (baobab)-[alberta@login2 rocker]$ salloc --partition=shared-cpu --time=00:30:00 --cpus-per-task 12
+  (baobab)-[alberta@cpu300 rocker]$ apptainer pull docker://rocker/rstudio:4.2
+  (baobab)-[alberta@cpu300 rocker]$ ls
+  rstudio_4.2.sif
+And Voila; I get my sif image rstudio_4.2.sif
 ==== Convert a Docker image ====

eResearch Doc

User Tools

Site Tools

Differences

Page Tools