Skip to content
Snippets Groups Projects
Commit 0f0504b8 authored by Oriol Tintó's avatar Oriol Tintó
Browse files

Update README.md

parent e7a4288f
No related branches found
No related tags found
No related merge requests found
Pipeline #18838 passed
......@@ -45,8 +45,7 @@ GIT:
We will modify this file as follows:
- Point to the proper branch ("master" should be stable)
- We will add a new variable to select which example we do want to run: `ICON_CASE: "real-from-ideal"`
- We will modify **CUSTOM_CONFIG** to point to the desired configurations.
- We will add a path in the local machine at which the output data will be transferred.
- We will modify **CUSTOM_CONFIG** to point to the desired configurations.
- We will modify **HPCARCH** to point to our desired platform.
```yaml
# Select case between real-from-ideal, real-from-dwd-ana
......@@ -75,15 +74,27 @@ GIT:
PROJECT_SUBMODULES: ""
FETCH_SINGLE_BRANCH: True
```
## 3. Additional configuration
We will create an additional yaml file with some more configuration.
> **_NOTE:_** It is important to add these things in a different file because **minimal.yml** is the first file to be read,
> and all the other configuration files overwrite its contents.
> If we try to replace a variable that also appears in the default configuration files (i.e. local_destination_folder) it won't have an effect.
The name of the file doesn't matter as long as it is a **.yml** file, for example **my_configuration.yaml**.
To this file we will add the following:
- We will add a path in the local machine at which the output data will be transferred.
```yaml
data_management:
# Where do we put the output files afterwards?
local_destination_folder: /Path/to/output/folder
```
- Some details about how to build icon.
## 3. Modify build command.
We are using spack to build icon. The default command that would be used to build icon would be:
Because we are using spack to build icon. The default command that would be used to build icon would be:
```bash
spack install icon-nwp@version%compiler
......@@ -94,32 +105,59 @@ spack install icon-nwp@version%compiler
However, in some systems it is necessary to link to specific libraries or use existing packages.
Some additional parameters need to be changed in the configuration to allow that.
Here we have two examples:
- At **LMU** we need to compile openmpi with a specific list of fabrics, and we need to use the external slurm as well. To do that we provide this additional configuration file:
- At **LMU**:
```yaml
spack:
compiler: "gcc@11.3.0" # desired compiler for spack
root: "$SCRATCH/autoicon-spack" # path to a spack install, will be downloaded to if not present
externals: "slurm"
user_cache_path: "$SCRATCH/autoicon-spackcache" # spack puts data here when bootstrapping, leave empty to use home folder
user_config_path: "$SCRATCH/autoicon-spackconfig" # spack puts data here when bootstrapping, leave empty to use home folder
disable_local_config: false # if true, spack installs into spack source dir
upstreams: "/software/opt/focal/x86_64/spack/2023.02/spack/opt/spack"
icon:
build_cmd: "icon-nwp@%ICON.VERSION%% %SPACK.COMPILER% ^openmpi+pmi+legacylaunchers schedulers=slurm fabrics=ucx ucx+dc+dm+ib_hw_tm+mlx5_dv+rc+rdmacm+thread_multiple+ud+verbs"
build_cmd: "icon-nwp@%ICON.VERSION%% %SPACK.COMPILER%+debug~mpichecks target=x86_64_v2 source=dkrz_https"
version: ${ICON_VERSION}
```
- At **LRZ** we rely on the spack package already available in the system that allows us to reuse most of the packages. To do so we add the following configuration:
- At **LRZ**:
```yaml
spack:
init: "module load user_spack" # command to load spack environment, e.g. module load spack, use spack/setup-env.sh if empty
root: "" # Setting this to an empty string to use the existing module.
compiler: gcc@11.2.0 # desired compiler for spack. Matching the one used for the existing packages in LRZ
compiler: "gcc@11.2.0" # desired compiler for spack
root: "$PLATFORM_SCRATCH/autoicon-spack" # path to a spack install, will be downloaded to if not present
externals: "slurm"
user_cache_path: "$PLATFORM_SCRATCH/autoicon-spackcache" # spack puts data here when bootstrapping, leave empty to use home folder
user_config_path: "$PLATFORM_SCRATCH/autoicon-spackconfig" # spack puts data here when bootstrapping, leave empty to use home folder
icon:
build_cmd: "icon-nwp@%ICON.VERSION%% %SPACK.COMPILER% ^openmpi/amct7nx"
build_cmd: "icon-nwp@%ICON.VERSION%% %SPACK.COMPILER%~mpichecks source=dkrz_https ^openmpi/amct7nx"
load_cmd: "icon-nwp@%ICON.VERSION%% %SPACK.COMPILER%~mpichecks source=dkrz_https"
version: 2.6.5-nwp0
```
> **_NOTE:_** At LRZ, the compute nodes don't have git available by default.
To overcome this problem we can compile the model and setup the python environment in the login nodes.
To do that, we can overwrite in which platform these jobs will be running by adding the following content to the file **my_configuration.yml**.
```yaml
Jobs:
BUILD_ICON:
PLATFORM: LRZ_LOGIN
BUILD_PYTHON_ENVIRONMENT:
PLATFORM: LRZ_LOGIN
```
## 4. Platforms
The user needs to provide a platforms file. It can be done in different ways but the easiest is to put the yaml file in the configuration folder.
Because we wanted to execute some of the lightweight runs on the login nodes of the remote platform, it is necessary to define not only the desired platform but also an additional login.
For example, if we use the platform **LMU**, we need to define **LMU_LOGIN** as well.
> **_NOTE:_** If we need to run in a login node (i.e. LRZ_LOGIN) we need to add this as a platform as well. Specifying type=ps .
## 5. Github password-less access
In order to get icon-sources through autosubmit, the password-less through ssh keys should be configured for dkrz gitlab.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment