Run Description File

Example Run Description YAML File

run id: SS-Atlantis

paths:
  atlantis code: /ocean/$USER/Atlantis/atlantis-trunk/
  atlantis executable name: atlantisMerged
  runs directory: /ocean/$USER/Atlantis/runs/
  atlantis command: $HOME/conda_envs/atlantis-dev/bin/atlantis

# The name of the boxes .bgm file must match the value of the "geometry" attribute in the
# global attributes of the initial conditions file.
boxes: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_xy.bgm

initial conditions: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_init.nc

groups: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_grps.csv

migrations: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SalishMigrations.csv

fisheries: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SalishFisheries.csv

parameters:
  # The run, forcing, physics, biology, and harvest parameters files are required for all runs,
  # and they must be identified by those exact keys.
  run: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_run.prm
  forcing: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_forcing.prm
  physics: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_physics.prm
  biology: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_biology.prm
  harvest: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_harvest.prm

forcing:
  # The keys must match the forcing file names in the forcing.prm file.
  # The file names in the forcing.prm file must be just file names, not path/filename.
  # The keys will be the names used for the symlinks created in the temporary run
  # directory.
  SS_hydro.nc:
    link to: /ocean/$USER/Atlantis/salish-sea-atlantis-model/inputs/SS_hydro.nc
  SS_temp.nc:
    link to: /ocean/$USER/Atlantis/salish-sea-atlantis-model/inputs/SS_temp.nc
  SS_salt.nc:
    link to: /ocean/$USER/Atlantis/salish-sea-atlantis-model/inputs/SS_salt.nc

output filename base: outputSalishSea

vcs revisions:
  git:
    - /ocean/$USER/Atlantis/salish-sea-atlantis-model/
    - /ocean/$USER/Atlantis/NEMO-Cmd/
    - /ocean/$USER/Atlantis/AtlantisCmd/

Basic Run Configuration

The following key-value pairs provide the basic configuration for the run.

An example of the basic configuration key-value pairs:

run id: SS-Atlantis

# The name of the boxes .bgm file must match the value of the "geometry" attribute in the
# global attributes of the initial conditions file.
boxes: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_xy.bgm

initial conditions: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_init.nc

groups: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_grps.csv

migrations: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SalishMigrations.csv

fisheries: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SalishFisheries.csv

output filename base: outputSalishSea

run id

An identifier for the run. It is used as the first part of the temporary run directory name. For example, run id: SS-Atlantis will result in the creation of a temporary run directory with a name like SS-Atlantis_2021-08-18T102810.108199-0700/. The date/time stamp that is appended to the run id value is the creation time of the directory. That makes the temporary run directory name unique, and allows atlantis run to be used to execute multiple nearly concurrent runs.

boxes

The path to the .bgm box geometry file for the run. The file name (without the path part) must match the value of the geometry global attribute in the initial conditions netCDF file.

initial conditions

The path to the initial conditions netCDF file that will be passed to the atlantisMerged executable by the -i flag.

groups

The path to the groups .csv file that will be passed to the atlantisMerged executable by the -s flag.

migrations

The path to the migrations .csv file that will be passed to the atlantisMerged executable by the -m flag.

fisheries

The path to the fisheries .csv file that will be passed to the atlantisMerged executable by the -q flag.

output filename base

The base filename for the output files generated by the run. For example, if the value is outputSalishSea, the main netCDF output file name will be outputSalishSea.nc, and other output file names will start with outputSalishSea; e.g. outputSalishSeaBoxBiomass.txt. The main netCDF output file name will be passed to the atlantisMerged executable by the -o flag.

The boxes, initial conditions, groups, migrations, and fisheries paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

Absolute paths with environment variables are strongly recommended for portability and re-usability.

Those files are copied to the temporary run directory during the run set-up phase of atlantis run, and they are copied from there to the results directory when the run script terminates so that they are preserved with the run results.

The file name of the boxes file in the temporary run directory and the results directory is the same as in the path provided. For example, boxes: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_xy.bgm will result in a file called SS_xy.bgm in the temporary run and results directories.

The file name of the initial conditions file in the temporary run directory and the results directory is always init_conditions.nc.

The file name of the groups file in the temporary run directory and the results directory is always groups.csv.

The file name of the migrations file in the temporary run directory and the results directory is always migrations.csv.

The file name of the fisheries file in the temporary run directory and the results directory is always fisheries.csv.

paths Section

The paths section of the run description file is a collection of directory paths that are used during the run set-up phase of atlantis run.

An example paths section:

paths:
  atlantis code: /ocean/$USER/Atlantis/atlantis-trunk/
  atlantis executable name: atlantisMerged
  runs directory: /ocean/$USER/Atlantis/runs/
  atlantis command: $HOME/conda_envs/atlantis-dev/bin/atlantis

atlantis code

The path to the CSIRO Atlantis ecosystem model code Subversion checkout where the Atlantis executable for the run is to be found.

The atlantis run sub-command confirms that the atlantis code directory exists, and exits with an error message if not.

atlantis executable name

The name of the Atlantis executable to use for the run. The executable name is typically atlantisMerged, but it may be different if you are with code that is under active development to add new features.

This is not actually a path, but it is included here because it is closely associated with atlantis code. The Atlantis executable is assumed to be located in the atlantis code tree, specifically, at {atlantis code}/atlantis/atlantismain/{atlantis executable name}.

The atlantis run sub-command confirms that the atlantis executable name file exists, and exits with an error message if not.

runs directory

The path to the directory where run directories will be created by the atlantis run sub-command.

atlantis command

The path to the atlantis command processor executable file installed from the AtlantisCmd Python package.

The atlantis run sub-command confirms that the atlantis command processor executable file exists, and exits with an error message if not.

The paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

Absolute paths with environment variables are strongly recommended for portability and re-usability.

parameters Section

The parameters section of the run description file is a collection of file paths that atlantis uses to find the .prm files for the run. The .prm files are copied to the temporary run directory during the run set-up phase of atlantis run, and they are copied from there to the results directory when the run script terminates so that they are preserved with the run results.

An example parameters section:

parameters:
  # The run, forcing, physics, and biology parameters files are required for all runs,
  # and they must be identified by those exact keys.
  run: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_run.prm
  forcing: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_forcing.prm
  physics: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_physics.prm
  biology: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_biology.prm
  harvest: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_harvest.prm

The paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

Absolute paths with environment variables are strongly recommended for portability and re-usability.

The 5 keys run, forcing, physics, biology, and harvest are required for all runs.

run

The path to the run parameters file that will be passed to the atlantisMerged executable by the -r flag. The file name of the run parameters file in the temporary run directory and the results directory is always run.prm.

forcing

The path to the forcing parameters file that will be passed to the atlantisMerged executable by the -f flag. The file name of the forcing parameters file in the temporary run directory and the results directory is always forcing.prm.

physics

The path to the physics parameters file that will be passed to the atlantisMerged executable by the -p flag. The file name of the physics parameters file in the temporary run directory and the results directory is always physics.prm.

biology

The path to the biology parameters file that will be passed to the atlantisMerged executable by the -b flag. The file name of the biology parameters file in the temporary run directory and the results directory is always biology.prm.

harvest

The path to the harvest parameters file that will be passed to the atlantisMerged executable by the -h flag. The file name of the harvest parameters file in the temporary run directory and the results directory is always harvest.prm.

Additional, optional, parameter file key-value pairs may be included as required. The file names of those extra parameters files in the temporary run directory and the results directory is the key with .prm appended; i.e. contaminants: /ocean/$USER/Atlantis/salish-sea-atlantis-model/SS_contaminant.prm would result in a file named contaminants.prm in the temporary run and results directories. Extra parameter files are generally associated with entries in the forcing parameter file.

forcing Section

The forcing section of the run description file contains sub-sections that provide the names of directories and files that are to be symlinked in the run directory for Atlantis to use to read forcing values from.

An example forcing section:

forcing:
  # The keys must match the forcing file names in the forcing.prm file.
  # The file names in the forcing.prm file must be just file names, not path/filename.
  # The keys will be the names used for the symlinks created in the temporary run
  # directory.
  SS_hydro.nc:
    link to: /ocean/$USER/Atlantis/salish-sea-atlantis-model/inputs/SS_hydro.nc
  SS_temp.nc:
    link to: /ocean/$USER/Atlantis/salish-sea-atlantis-model/inputs/SS_temp.nc
  SS_salt.nc:
    link to: /ocean/$USER/Atlantis/salish-sea-atlantis-model/inputs/SS_salt.nc

The sub-section keys (SS_hydro.nc, SS_temp.nc, and SS_salt.nc above) are the names of the symlinks that will be created in the run directory. Those names are expected to appear in the appropriate places in the forcing.prm file. The values associated with the link to keys are the targets of the symlinks that will be created in the run directory.

The paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

Absolute paths with environment variables are strongly recommended for portability and re-usability.

You are free to use any keys that you wish with the understanding that the key will be the name of the symlink that will be created in the run directory, and that name will also need to appear as a file name in the forcing.prm file.

The atlantis run command confirms that the targets of the symlinks exist, and exits with an error message if not.

vcs revisions Section

The optional vcs revisions section of the run description file contains lists of version control system repositories for which the revision and status will be recorded in the temporary run and run results directories.

Note

Generation of a revision and status record file for the atlantis-trunk Subversion repository checkout is not yet implemented.

An example vcs revisions section:

vcs revisions:
  git:
    - /ocean/$USER/Atlantis/salish-sea-atlantis-model/
    - /ocean/$USER/Atlantis/NEMO-Cmd/
    - /ocean/$USER/Atlantis/AtlantisCmd/

The sub-section key(s) (git above) are the names of the version control tools to use for the repositories listed below them. At present only Git (git) and Mercurial (hg) are supported.

The paths listed under the version control tool keys are the repositories of which the revision and status will be recorded.

The repository paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

Absolute paths with environment variables are strongly recommended for portability and re-usability.

For each repository, a file will be created in the temporary run directory. The file names are the repository directory names with _rev.txt appended. So, from the example above, the files created will be:

salish-sea-atlantis-model_rev.txt
NEMO-Cmd_rev.txt
AtlantisCmd_rev.txt

For Git repositories, each _rev.txt file will contain the output of the commands:

git branch --show-current
git log -1
git show --pretty="" --name-only

for the repository. That is a record of the last committed revision of the repository that will be in effect for the run. For example, salish-sea-atlantis-model_rev.txt might contain:

branch: master
commit: 9c87f2789a6fe4acfa84545d0d243b404dda9dbe
author: por07g
date:   Wed Jun 09 09:32:38 2021 +10:00
files:  RunTrack.org
message:
update

If any of the listed repositories contain uncommitted changes, the nemo prepare command that atlantis run uses will generate a warning message like:

nemo_cmd.prepare WARNING: There are uncommitted changes in /ocean/$USER/Atlantis/salish-sea-atlantis-model/

and the list of uncommitted changes and their status codes, the output of the git diff --name-status command, will be appended to the _rev.txt file. Example:

branch: master
commit: 9c87f2789a6fe4acfa84545d0d243b404dda9dbe
author: por07g
date:   Wed Jun 09 09:32:38 2021 +10:00
files:  RunTrack.org
message:
update

uncommitted changes:
M SS_forcing.prm
M SS_run.prm