Testing¶

The suite has Rust unit/integration tests for Ark, Ark Core, the API, the Cube Coordinator and the distributed protocol, plus Python binding tests for Ark and Ark Core. CI runs them on pushes to main and on internal pull requests, limited to the crates a change touches.

Pre-release checks¶

Use the repository-pinned Rust toolchain and a fresh Python 3.12 virtual environment for CI parity. Install requirements-test.txt: the script tests need NumPy even though the published Ark wheel has no Python runtime dependencies.

Run these before merging or releasing — these mirror the correctness jobs in CI:

.venv/bin/python -m pip install -r requirements-test.txt
.venv/bin/python -m maturin develop --manifest-path python/Cargo.toml

# Formatting and lint (Rust 1.98.0)
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings

# Ark Core (CI: scripts/ci_rust_tests.py core / core-features / core-all-features)
cargo test --locked -p patronus-ark-core --no-default-features
cargo test --locked -p patronus-ark-core --features wolf
cargo test --locked -p patronus-ark-core --features download
cargo test --locked -p patronus-ark-core --all-features --lib

# Rust unit + integration tests
cargo test -p patronus-ark
cargo test -p ark-api
cargo test --locked -p ark-coordinator
cargo test --locked -p ark-distributed-protocol

# Python binding and benchmark-script tests
.venv/bin/python -m unittest discover -s python/tests
.venv/bin/python -m pytest scripts/tests -q
.venv/bin/python scripts/generate_docs.py --check

# Ark Core Python package (CI: core-python.yml, on a release wheel)
.venv/bin/python -m maturin develop --manifest-path ark-core/python/Cargo.toml
.venv/bin/python -m pytest ark-core/python/tests -q
cargo deny check licenses

What the tests cover¶

Rust (`rust/tests/`)¶

File	Focus
`native_detectors.rs`	Native L1 detector behavior.
`governance_attack_regressions.rs`	Injection L1 governance attacks stay terminal with their audited rule; nearby benign forms stay nonterminal.
`pipeline_units.rs`, `pipeline_routing.rs`	Pipeline composition and per-category routing.
`ml_units.rs`	ML/ONNX units.
`l3_scheduler.rs`	L3 worker scheduling, fairness, max-wait.
`unified_l3.rs`	Unified multi-head L3 behavior.
`dynamic_pii.rs`	GLiNER dynamic-PII pipeline.
`shell_command.rs`	`shell_command` gateway wiring: bundle checks, manifest pin, user `.cedar` policies.
`shell_risk_parity.rs`	K26 shell-risk verdict parity with the Python reference (`kf1`, facts, policies, scores, verdict).
`assets_manifest.rs`	Asset manifest integrity.
`hf_e2e.rs`	End-to-end with real Hugging Face assets (needs network / `HF_TOKEN`).

Python (`python/tests/`)¶

File	Focus
`test_public_api.py`	The public `SecurityGateway` surface and result shapes.
`test_native_scanners.py`	Native scanning through the bindings.
`test_benchmark_data.py`	Integrity of the shipped benchmark validation samples.

Ark Core¶

Unit tests live next to the code in ark-core/src/ (detectors, threat rules, Wolf, Cat, scanner). ark-core/python/tests/ (test_scanner.py, test_concurrency.py) covers the patronus_ark_core Python package. ark-coordinator and ark-distributed-protocol keep their tests in their crates.

Token pipeline (no inference)¶

cargo test -p patronus-ark --lib token_pipeline_e2e exercises compact tokenization, v4 L2-output materialization, promotion, duplicate-head handoff, and L3 input assembly without loading either model. cargo test -p patronus-ark --lib ml::tokenizer also checks input byte limits, exactly-once window encoding, true source offsets, BOS/EOS, padding, and rejection instead of truncation or Hugging Face fallback. These tests use the small checked-in compact fixture and run in normal CI.

The ignored mmbert_mmbpe_matches_huggingface_ids_and_special_tokens test additionally compares the real compact artifact's IDs and source offsets with its conversion source, then verifies token-only L3 input assembly. Set PATRONUS_TEST_MMBERT_TOKENIZER_JSON and PATRONUS_TEST_MMBERT_TOKENIZER_MMBPE to the pinned local tokenizer artifacts and run that test with -- --ignored. It performs no model inference.

Network-dependent tests¶

hf_e2e.rs and anything that exercises L2/L3 needs model assets. Provide HF_TOKEN if the repositories require authentication. Tests that only exercise native L1 run fully offline. The real-download smoke test hf_shell_command_download_warmup_smoke (in hf_e2e.rs) runs only with PATRONUS_RUN_HF_E2E=1.

Benchmarking (not a correctness gate)¶

The local benchmark measures accuracy, latency, and false-positive rate on the shipped samples. It is a measurement tool, not a pass/fail test — use it to validate performance changes, not as part of the PR gate. Do not commit benchmark output (it is machine-specific and git-ignored).

Separately, CI's native-throughput job installs a fresh release wheel and runs scripts/ark_api_throughput_benchmark.py against the real L1/L2/unified-L3 profile, without GLiNER. Its 0.3 MiB/s and 0.5 requests/s limits are pass/fail gates; a local macOS result does not prove that the Linux runner passes them.

Adding tests¶

New native detector → add cases to native_detectors.rs and, if exposed, test_native_scanners.py.
New public API → extend test_public_api.py so the result shape is pinned.
Threshold/manifest/result-shape change → update the affected tests and document it in the changelog.