The benchmark for the compute economy.

It's underappreciated how much compute quality varies, and that matters more than ever now that hundreds of new GPU providers are coming online. We're building an open benchmark for compute quality.

Published GPU specifications generalize across silicon and often overlook details that matter in practice, like the silicon lottery, heterogeneous power and networking setups, or cooling. We measure nodes in the context of what they're supposed to do, covering both bare-metal performance and managed clusters. Measurements include synchronized training steps, collective communication, sustained matmul throughput, and checkpoint I/O under load.

Our checks measure every node against a baseline tailored to that specific GPU rather than a threshold copied off a datasheet, then go deeper into fleet-specific health. We catch stragglers before one slow rank sets the pace of a synchronized job, attribute slowdowns to the component responsible, and flag a card that is merely below par as clearly as one that is broken.

• Hardware healthDriver and device visibility, ECC health, CPU and memory sanity, NUMA affinity, essential daemons, clock sync, filesystem availability, and temperature and fan behaviour.
• Communication reliabilityNVLink and PCIe link health, NCCL collectives, NIC link health, RoCE checks, and inter-node bandwidth.
• Performance consistencySustained stress, matmul FLOPS, memory bandwidth, short training probes, multi-node bandwidth variance, checkpoint I/O performance, throttling under sustained load, and cooldown profiles.
• Numerical correctnessSilent-data-corruption checks against known-answer references.

We provide coverage across NVIDIA and AMD, on Slurm, Kubernetes, or plain SSH, for testing at handoff, burn-in, and periodic recertification.