How Sixtytwo tests your nodes

We qualify real GPU hosts so the capacity you rent or connect is the capacity that actually works.

Sixtytwo runs automated tests and short reference workloads to produce a per-node trust score. The goal is simple: catch unhealthy hardware before a job starts, spot problems while it runs, and keep a history that makes future decisions more reliable.

What we test

• GPU: driver and device visibility, ECC health, temperature and fan, memory bandwidth.
• Compute: sustained stress, matmul FLOPS, short training probes and silent-data-corruption checks.
• Interconnect: NVLink/PCIe link health, NCCL collectives, multi-node bandwidth and variance.
• Host: CPU/memory sanity, NUMA affinity, essential daemons and clock sync.
• Network: NIC link health and inter-node bandwidth / RoCE checks.
• Storage: filesystem availability and checkpoint I/O performance.

How deep

Trust score

Every run updates a node’s trust score. We combine recent test results, runtime fault events, and recovery history in a recency-weighted Bayesian update, so the score reflects both current hardware health and recent track record, with more recent evidence counting for more. A high score means the node is safe to prioritize for long jobs; a low one pulls it out of scheduling until it proves healthy again.

Example report

Node: gpu-04 | Result: PASS (1 WARN) | Trust 0.87
GPU: PASS (FP16 perf ok, ECC clean)
Compute: PASS (short training probe)
Interconnect: WARN (NVLink bandwidth below fleet median)

Benchmarking providers

Our cloud performance reports run this same suite on rented capacity. Three additional notes for our reports:

Node class. A full-access rental (bare metal, or a VM where you own the OS) exposes the host: DIMM inventory, host DRAM ECC, the IPMI/BMC event log, NUMA tuning, the fabric-manager service. A container rental, the common shape of budget on-demand and spot capacity, only exposes what the runtime passes through. Most checks run in both. The host-only checks report as not applicable on containers and are excluded from scoring, and every run is labeled with its node class, so a container is never scored against checks it cannot see.

Remediation. Where a report covers fault handling, we induce a single reversible fault on a node we are renting and measure whether the provider detects and replaces it, and how quickly. This runs only with the provider's explicit consent.

Run it yourself. The suite ships on PyPI as sixtytwo-cli. It needs Python 3.11 or newer, so on a fresh GPU node the reliable install is uv, which brings its own Python:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv ~/sxty-env --python 3.12
uv pip install --python ~/sxty-env/bin/python 'sixtytwo-cli[gpu]'

# the same checks the reports are built from
~/sxty-env/bin/sixtytwo init
~/sxty-env/bin/sixtytwo test --full --json > report.json

Interested?

Reach out at founders@sixtytwo.ai.