AutoIcon

Available examples:

• real-from-ideal: Start with a test simulation (ltestcase=.TRUE.) to produce a first-guess file and an analysis file to start a second simulation.
• real-from-dwd-ana: Start with a first-guess and analysis from DWD (relying on the local archive to do that.)

NOTE: The current implementation of real-from-dwd-ana relies on the local archives at LMU. The templates to provide the initial conditions have to be modified to run from a different local system.

With autosubmit installed, we can run these ICON workflows with autosubmit we need to do the following:

1. Create an experiment:

```bash
autosubmit expid -min -repo  https://gitlab.physik.uni-muenchen.de/w2w/autoicon.git -d "Launching an ICON experiment with autosubmit."
```

This will create a new experiment folder and will give us the corresponding experiment id.

2. Modify the minimal.yml configuration file

If we go to the **conf** folder corresponding to our experiment, we will find a **minimal.yml** file there:

CONFIG:
  AUTOSUBMIT_VERSION: "4.0.76"
  TOTALJOBS: 20
  MAXWAITINGJOBS: 20
DEFAULT:
  EXPID: "a00p"
  HPCARCH: "local"
  #hint: use %PROJDIR% to point to the project folder (where the project is cloned)
  CUSTOM_CONFIG: "%PROJDIR%/"
PROJECT:
  PROJECT_TYPE: "git"
  PROJECT_DESTINATION: "git_project"
GIT:
  PROJECT_ORIGIN: "https://gitlab.physik.uni-muenchen.de/w2w/autoicon.git"
  PROJECT_BRANCH: ""
  PROJECT_COMMIT: ""
  PROJECT_SUBMODULES: ""
  FETCH_SINGLE_BRANCH: True

We will modify this file as follows:

• Point to the proper branch ("master" should be stable)
• We will add a new variable to select which example we do want to run: ICON_CASE: "real-from-ideal"
• We will modify CUSTOM_CONFIG to point to the desired configurations.
• We will add a path in the local machine at which the output data will be transferred.
• We will modify HPCARCH to point to our desired platform.

# Select case between real-from-ideal, real-from-dwd-ana
ICON_CASE: "real-from-ideal"

CONFIG:
  AUTOSUBMIT_VERSION: "4.0.76"
  TOTALJOBS: 20
  MAXWAITINGJOBS: 20
DEFAULT:
  EXPID: "a00p"   # Will depend on the actual experiment id.
  HPCARCH: "LRZ"
  #hint: use %PROJDIR% to point to the project folder (where the project is cloned)
  CUSTOM_CONFIG:
    PRE:
      - "%PROJDIR%/conf/common"
      - "%PROJDIR%/conf/%ICON_CASE%"

PROJECT:
  PROJECT_TYPE: "git"
  PROJECT_DESTINATION: "git_project"
GIT:
  PROJECT_ORIGIN: "https://gitlab.physik.uni-muenchen.de/w2w/autoicon.git"
  PROJECT_BRANCH: "master"
  PROJECT_COMMIT: ""
  PROJECT_SUBMODULES: ""
  FETCH_SINGLE_BRANCH: True

data_management:
  # Where do we put the output files afterwards?
  local_destination_folder: /Path/to/output/folder

3. Modify build command.

We are using spack to build icon. The default command that would be used to build icon would be:

spack install icon-nwp@version%compiler

NOTE: Since in the LMU example we are building everything from scratch, probably it is the best starting point to try to build icon with spack in a new platform.

However, in some systems it is necessary to link to specific libraries or use existing packages. Some additional parameters need to be changed in the configuration to allow that. Here we have two examples:

• At LMU we need to compile openmpi with a specific list of fabrics, and we need to use the external slurm as well. To do that we provide this additional configuration file:

spack:
  externals: "slurm"
icon:
  build_cmd: "icon-nwp@%ICON.VERSION%% %SPACK.COMPILER% ^openmpi+pmi+legacylaunchers schedulers=slurm fabrics=ucx ucx+dc+dm+ib_hw_tm+mlx5_dv+rc+rdmacm+thread_multiple+ud+verbs"

• At LRZ we rely on the spack package already available in the system that allows us to reuse most of the packages. To do so we add the following configuration:

spack:
  init: "module load user_spack" # command to load spack environment, e.g. module load spack, use spack/setup-env.sh if empty
  root: "" # Setting this to an empty string to use the existing module.
  compiler: gcc@11.2.0 # desired compiler for spack. Matching the one used for the existing packages in LRZ

icon:
  build_cmd: "icon-nwp@%ICON.VERSION%% %SPACK.COMPILER% ^openmpi/amct7nx"

4. Platforms

The user needs to provide a platforms file. It can be done in different ways but the easiest is to put the yaml file in the configuration folder. Because we wanted to execute some of the lightweight runs on the login nodes of the remote platform, it is necessary to define not only the desired platform but also an additional login. For example, if we use the platform LMU, we need to define LMU_LOGIN as well.

5. Github password-less access

In order to get icon-sources through autosubmit, the password-less through ssh keys should be configured for dkrz gitlab.

6. Create and run

With this changes in the configuration should be already possible to create the experiment and run it.