Testing
GKX tests protect contracts, not line counts. A test belongs in the suite when it checks one of:
an implemented equation or a reduced physical limit;
a numerical method: convergence order, conservation, free-energy identity;
a geometry, normalization, or diagnostic convention;
a benchmark artifact and its documented fit/window policy;
an autodiff contract against finite differences, tangents, or an adjoint identity;
a regression for a bug found in parity, restart, runtime, plotting, or geometry-adapter work.
The package-wide coverage target is 95%, enforced on the combined wide-coverage run (see CI jobs). Coverage is a guardrail; the physics and numerics contracts above are the objective.
Test tree
tests/README.md owns the layout. Do not add flat tests/test_*.py files;
prefer one parametrized module per physical or workflow contract.
Directory |
Contents |
|---|---|
|
Fast contracts for package code, by domain: |
|
Executable behavior: runtime TOML loading and runs
( |
|
Literature-anchored physics gates ( |
|
Tests for |
|
Repository-policy gates ( |
|
Shared helpers; not package API. |
Representative contracts and where they live:
Hermite/Laguerre ladder identities, quasineutrality, and streaming (
gkx.operators.linear.moments):tests/unit/linear/test_linear.py,tests/unit/linear/test_linear.py,tests/unit/operators/test_terms_fields.py.Term-wise RHS equivalence (
gkx.terms.assemble_rhs_cachedagainstgkx.operators.linear.rhs.linear_rhs_cached):tests/unit/operators/test_terms_fields.py.Term toggles, drift activation, diamagnetic drive vanishing at \(k_y=0\),
rho_starscaling of cached \(k_y\), and the end-damping taper:tests/unit/linear/test_linear.py.Collision matrices and free-energy identities, end damping, Hermite hierarchy, geometry conventions:
tests/validation/physics_gates/.Growth-rate fit windows (
gkx.diagnostics.growth_rates):tests/unit/diagnostics/test_analysis.py.Krylov/shift-invert internals, mode-family targeting, fallback policy:
tests/unit/solvers/test_linear_krylov_core.py.Nonlinear bracket sign, real-FFT path, flutter, electromagnetic split:
tests/unit/nonlinear/test_nonlinear_exb.py; Hermitian projection, RK variants, collision splitting, IMEX, flow-shear remap and window-gradient checks:tests/unit/nonlinear/test_nonlinear_helpers_extra.py.Serial-vs-parallel numerical identity:
tests/unit/parallel/; the tracked large-run scaling artifacts:tests/unit/parallel/test_parallel_core.py.
Running tests
JAX_ENABLE_X64=true GKX_X64=1 pytest
pytest.ini sets addopts = -q --maxfail=1 --disable-warnings -m "not
slow", so a bare pytest runs everything except tests marked slow.
CI sets JAX_ENABLE_X64=true and GKX_X64=1 for every test step except the
two float32 steps in python-floor; set them locally when reproducing CI.
Markers:
slow: excluded by default and by the wide-coverage shards, so aslowtest leaves the coverage denominator. Do not use it for a test that only takes a minute.integration: a label for end-to-end tests; it does not deselect anything.gpu: exercises a device-specific execution path.
For a bounded local loop, use the per-file runner. It applies a per-file
timeout and a whole-run timeout (both 300 s by default) and reports files it
did not reach as not_run(total_timeout):
python scripts/checks/run_test_gates.py fast
python scripts/checks/run_test_gates.py fast --total-timeout 0 # full sequential pass
Opt-in parity gates skip unless their environment variable is set:
# restart parity: resumed nonlinear run equals the continuous run
pytest -q tests/integration/runtime/test_runtime_runner.py -k restart_gate
# CPU vs GPU short nonlinear trajectory
GKX_DEVICE_PARITY=1 pytest -q tests/unit/parallel/test_parallel_core.py -k cpu_gpu
# VMEC -> *.eik.nc regenerated twice, arrays bitwise identical
GKX_VMEC_FILE=/path/to/wout.nc pytest -q tests/unit/geometry/test_geometry.py -k roundtrip
CI jobs
.github/workflows/ci.yml runs these jobs on every push and pull request.
All Python jobs use 3.11, except python-floor (3.10), and install through .github/actions/setup: uv
with a cached wheel store, and, for the test jobs, a JAX persistent
compilation cache restored per job and saved whenever src/ or tests/
change. Each job installs for itself; the only needs: edge is
wide-coverage on wide-coverage-shards.
Job |
What it runs |
|---|---|
|
|
|
Installs at |
|
|
|
Paired eigenmode and implicit-AD gates against the released SOLVAX
( |
|
Release-surface files; requires >= 95% line coverage on |
|
The test lane: 8 shards of |
|
Combines the shard data (every shard must report), enforces >= 95%
package-wide, and writes
|
|
Coverage-manifest summary, quasilinear |
|
Runs with |
|
Only on |
Core coverage gates in nightly-full:
# gkx.terms >= 90%
pytest -q tests/unit/operators/test_terms_fields.py \
tests/unit/operators/test_terms_fields.py \
tests/unit/operators/test_terms_fields.py \
tests/unit/solvers/test_time_integrators.py \
tests/unit/nonlinear/test_nonlinear_exb.py \
tests/unit/nonlinear/test_nonlinear.py \
--cov=src/gkx/terms --cov-fail-under=90
# solvers_linear_krylov.py >= 90%, read from coverage-core.xml
pytest -q tests/unit/solvers/test_linear_krylov_core.py \
--cov=src/gkx --cov-report=xml:coverage-core.xml
The wide gate runs locally with the same helper. One process:
python scripts/checks/run_test_gates.py wide-coverage \
--shards 8 --workers 4 --timeout 1800 --fail-under 95 \
--pytest-arg=-o --pytest-arg=addopts= \
--pytest-arg=-m --pytest-arg="not slow"
Or shard by shard, then combine, as CI does:
python -m coverage erase
for shard in $(seq 1 8); do
python scripts/checks/run_test_gates.py wide-coverage \
--shards 8 --workers 4 --timeout 1800 --only-shard "${shard}" \
--keep-existing-coverage --skip-combine \
--pytest-arg=-o --pytest-arg=addopts= \
--pytest-arg=-m --pytest-arg="not slow"
done
python scripts/checks/run_test_gates.py wide-coverage \
--shards 8 --combine-only --fail-under 95
A shard that owns a logical-CPU file runs all of its device gates at four devices; a lone file in a command is spread test by test over the workers, several files are spread file by file so each keeps its in-process compiles.
Contracts a local pytest cannot catch
These are enforced by manifest scripts, the docs build, or a different interpreter. Run them before pushing.
Repo-wide line budget. The physical line count of
src/gkx/**/*.pymay not exceed theinstallable_source_python_linesbaseline intools/package_architecture_manifest.toml. To raise it, edit the baseline with an inline# old -> new: reasoncomment.Per-module cap.
[complexity_policy]in the same manifest setsdefault_max_lines = 1200per module andpublic_facade_max_lines = 500for the listed facades. Going over needs a reviewed[[complexity_policy.exceptions]]entry.Coverage-owner manifest. Every module under
src/gkxmust appear intools/validation_coverage_manifest.tomlas a direct[[modules]]row, inside a row’sowned_modules, or incoverage_inventory.excluded_modules. Modules of 2000 or more source lines need a direct row.fast_testsandartifact_pathsname files, not directories, and no list repeats an entry.API reference. Every
.. automodule:: gkx...indocs/api.rstmust resolve to a real module tracked by the coverage manifest, and the docs build with-W.Python floor.
python-floorcollects the suite on 3.10 and runs a bounded unit subset there; TOML is read withtomllib, falling back to thetomlibackport on 3.10, and a release gate keeps that guarded form the only one outside the tests.
The checklist in CI order:
python scripts/checks/check_repository_size_manifest.py
python scripts/checks/check_package_architecture_manifest.py
python scripts/checks/check_validation_coverage_manifest.py
mypy
pytest -q --collect-only --disable-warnings > /dev/null
pytest tests/release/test_release_gates.py
python -m sphinx -W -b html docs docs/_build/html
Focused checks for items 3 and 4:
pytest tests/release/test_release_gates.py \
-k "documented_public_api or large_modules_have_direct_manifest_rows"
Numerical stack
pyproject.toml requires jax>=0.10.1, jaxlib>=0.10.1, and
solvax>=0.29.0 on Python 3.11+, and jax>=0.6.2 on Python 3.10, where
0.6.2 is the newest JAX with wheels; a few reverse-mode tests are skipped there.
Every CI job prints the resolved jax,
jaxlib, and numpy versions (“Print benchmark stack”); reproduce a parity
regression on that stack, with JAX_ENABLE_X64=1, before changing solver
logic. Near-marginal lanes (TEM, ETG scans, some imported-linear stellarator
cases) can move under a different JAX/NumPy combination.
Validation gate tooling
Artifact gates share one JSON report convention (observable, reference,
absolute/relative tolerance, pass/fail) from
scripts.checks._gates.validation_gates: evaluate_scalar_gate,
observed_order_gate_report, branch_continuity_gate_report,
eigenfunction_gate_report, linear_metrics_gate_report,
nonlinear_window_gate_report, and zonal_response_gate_report. Artifacts
that set gate_index_include=false are documented diagnostics, not release
gates.
Command |
Artifact and current result |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Checks every |
|
Selects only a passing broad matrix family; current negative ledger:
|
|
Admits an external-VMEC holdout at high grid only when the low-grid
failure is the sole failure and time-horizon and replicate gates pass;
it does not claim full |
|
|
check_quasilinear_promotion_guardrails.py fails if a promoted quasilinear
report lacks train/holdout points, finite nonlinear window statistics, a
passed holdout gate, or calibration-policy metadata, and it scans the
claim-scope wording in the README and docs. This is not a runtime/TOML
absolute-flux predictor; it is a fast metadata and wording guard.
Every shipped TOML that reaches the nonlinear runtime either pins run_to in
its [time] block or is listed in tests/release/test_release_gates.py
with its measured first-chunk stop decision. A zero-gradient relaxation run
under the default run_to = "saturation" would stop in the first chunk, so
the Merlo deck pins run_to = "t_max" and its artifact gates
trace_completeness = 1.