GPU-native regional ocean solver — Arakawa C-grid, continuity-PPM, split-explicit RK2, ALE vertical coordinates, portable across NVHPC/gfortran/ifx.

Find us on…

GitHub Download the Source

Roundabout

Roundabout is a GPU-native Fortran solver for regional and global hydrostatic ocean: a structured-Cartesian Arakawa C-grid dynamical core (continuity-PPM, PV-conserving Coriolis, split-explicit RK2, ALE vertical coordinates) with sea ice.

GPU parallelism is expressed with do concurrent + OpenACC, portable across NVHPC (GPU and multicore), gfortran, and ifx.

Scope

sim_type='ocean' is the only regime this build ships — the coastal A-grid (HLL/HLLC) path and the unstructured triangular backend were carved out into their own repository.

The full architectural overview, repository map, and run lifecycle live in docs/codebase/INDEX.md; the god-state slot map and design rules are in src/core/ocean/README.md.

Building

module load cmake nvhpc
cmake -B build -S .
cmake --build build
cd build && ctest --output-on-failure

Key CMake options:

Option Default Description
RDB_ENABLE_GPU OFF OpenACC GPU offload via NVHPC (opt-in)
RDB_ENABLE_MPI OFF MPI multi-rank support
RDB_CUDA_AWARE_MPI OFF GPU-direct halo exchange (requires CUDA-aware MPI)
RDB_GPU_ARCH cc70 Target GPU compute capability (cc70/80/90)
RDB_ENABLE_DOUBLE ON Double-precision working precision

The full list lives in CMakeLists.txt.

Do not pass -j N to ctest when the GPU build is active — every worker shares one GPU, so parallel test execution produces spurious failures and hangs. Building with -j is fine; only ctest -j N is the problem. Use ctest -R rdb for the ~164-test rdb-only subset (~60 s) for fast regression checks.

Documentation

Per-module and per-procedure documentation is generated by FORD from !! docstrings in the source — that is the canonical reference. Hand-maintained synthesis documents complement it:

Comments prefixed with !! are documentation comments processed by FORD. Comments prefixed with a single ! are ordinary code comments and are not rendered. Please do not use !! for casual code comments.

Dependencies

  • Fortran compiler — NVHPC (GPU + multicore), gfortran, or ifx.
  • CMake ≥ 3.22.
  • NetCDF-Fortran.
  • An MPI implementation (only when RDB_ENABLE_MPI=ON; OFF builds pic-mpi’s serial backend instead and needs no MPI library).
  • Internet access at configure time to pull pic, pic-mpi, and test-drive.

Pre-commit workflow

Before each commit, in order:

  1. pre-commit run --all — formatting (whitespace, trailing newlines, end-of-file).
  2. fortitude check — Fortran static checks (also wired up as a pre-commit hook).
  3. ctest — full test suite. Never pass -j N on the GPU build.
  4. Re-read the synthesis docs (CLOSURE_MATRIX.md, the design-contract READMEs, docs/howto/, CAPABILITIES_AND_LIMITATIONS.md) and update anything that drifted. When code and docs disagree, the code is authoritative.

When adding a new capability: namelist knob (default off, preserves bit-identity) → kernel gated on the knob → unit test (analytical tests are high-leverage) → format/lint/test → one commit per capability, one PR per capability.

Contributing

Contributions are welcome. Please follow the conventions documented in FORTRAN_STYLE.md and the design contract in src/core/ocean/README.md. File a PR against main — FORD docs are rebuilt on every PR merged to main.

License

See LICENSE in the source tree.

Developer Info

Jorge Luis Galvez Vallejo