GPU-native regional ocean solver — Arakawa C-grid, continuity-PPM, split-explicit RK2, ALE vertical coordinates, portable across NVHPC/gfortran/ifx.
Find us on…
Roundabout is a GPU-native Fortran solver for regional and global hydrostatic ocean: a structured-Cartesian Arakawa C-grid dynamical core (continuity-PPM, PV-conserving Coriolis, split-explicit RK2, ALE vertical coordinates) with sea ice.
GPU parallelism is expressed with do concurrent + OpenACC, portable across NVHPC (GPU and multicore), gfortran, and ifx.
sim_type='ocean' is the only regime this build ships — the coastal A-grid
(HLL/HLLC) path and the unstructured triangular backend were carved out into
their own repository.
The full architectural overview, repository map, and run lifecycle live in docs/codebase/INDEX.md; the god-state slot map and design rules are in src/core/ocean/README.md.
module load cmake nvhpc
cmake -B build -S .
cmake --build build
cd build && ctest --output-on-failure
Key CMake options:
| Option | Default | Description |
|---|---|---|
RDB_ENABLE_GPU |
OFF |
OpenACC GPU offload via NVHPC (opt-in) |
RDB_ENABLE_MPI |
OFF |
MPI multi-rank support |
RDB_CUDA_AWARE_MPI |
OFF |
GPU-direct halo exchange (requires CUDA-aware MPI) |
RDB_GPU_ARCH |
cc70 |
Target GPU compute capability (cc70/80/90) |
RDB_ENABLE_DOUBLE |
ON |
Double-precision working precision |
The full list lives in CMakeLists.txt.
Do not pass
-j Ntoctestwhen the GPU build is active — every worker shares one GPU, so parallel test execution produces spurious failures and hangs. Building with-jis fine; onlyctest -j Nis the problem. Usectest -R rdbfor the ~164-test rdb-only subset (~60 s) for fast regression checks.
Per-module and per-procedure documentation is generated by FORD from !! docstrings in the source — that is the canonical reference. Hand-maintained synthesis documents complement it:
docs/CLOSURE_MATRIX.md — which closure/scheme is active in which regime, plus tunable knobs (drift-checked by tools/check_closure_matrix.py).docs/CAPABILITIES_AND_LIMITATIONS.md — what is shipped, what is deferred.docs/REFERENCE.md — physics + namelist reference.FORTRAN_STYLE.md — coding style and the do concurrent + OpenACC GPU programming model.docs/codebase/CONCEPTS.md — glossary of acronyms.docs/codebase/INDEX.md — repository map and run lifecycle.Comments prefixed with !! are documentation comments processed by FORD. Comments prefixed with a single ! are ordinary code comments and are not rendered. Please do not use !! for casual code comments.
RDB_ENABLE_MPI=ON; OFF builds
pic-mpi’s serial backend instead and needs no MPI library).pic, pic-mpi, and test-drive.Before each commit, in order:
pre-commit run --all — formatting (whitespace, trailing newlines, end-of-file).fortitude check — Fortran static checks (also wired up as a pre-commit hook).ctest — full test suite. Never pass -j N on the GPU build.CLOSURE_MATRIX.md, the design-contract READMEs, docs/howto/, CAPABILITIES_AND_LIMITATIONS.md) and update anything that drifted. When code and docs disagree, the code is authoritative.When adding a new capability: namelist knob (default off, preserves bit-identity) → kernel gated on the knob → unit test (analytical tests are high-leverage) → format/lint/test → one commit per capability, one PR per capability.
Contributions are welcome. Please follow the conventions documented in FORTRAN_STYLE.md and the design contract in src/core/ocean/README.md. File a PR against main — FORD docs are rebuilt on every PR merged to main.
See LICENSE in the source tree.