Checkpointing

Saving and loading model state at scale: sharded arrays, multihost clusters, and reading exactly the bytes each machine needs.

Explainers

• Loading a Safetensors Checkpoint on a Multihost Cluster, Line by Line — A safetensors file is one flat byte blob per tensor, but a big model must land as sharded jax.Arrays across many hosts. Walk the real loader that maps each host’s shards to byte ranges — and reads exactly those.