Execute Vast.ai primary workflow: Core Workflow A. Use when implementing primary use case, building main features, or core integration tasks. Trigger with phrases like "vastai main...
Treat a training run as a recoverable state machine, not an SSH session. Bind code and image identity, select an offer through policy, persist checkpoints outside the disposable root disk, export final evidence, and destroy.
Record code revision, image digest, dataset version, command, seed, expected checkpoint cadence, budget, and output destination.
Search only verified rentable offers that meet the manifest, record price components, and create one labeled instance. Persist new_contract immediately.
Poll structured instance state with a deadline and terminal branches. Confirm image identity, disk headroom, GPU model, and CUDA visibility.
Start the workload so checkpoints are uploaded or copied to durable storage at the declared cadence. A local checkpoint alone is not recovery evidence.
Validate artifact checksums and run metadata. For an interruption, provision a replacement from policy and prove resume from the last durable checkpoint.
Copy final logs and checksums, destroy the instance, confirm removal, and reconcile actual spend against the manifest.
Use a scoped control-plane key for search and instance operations. Give the workload only the storage credential needed for its checkpoint prefix, with no Vast.ai billing or team authority.
Use Read and Grep to inspect manifests, configuration, provider output, and existing tests before proposing a mutation. Use Write or Edit only for the approved plan, implementation, test, or redacted receipt; do not create, update, destroy, or fund Vast.ai resources without explicit operator approval.
Return revision, image digest, offer and instance IDs, checkpoint URI/checksum, terminal result, actual spend, and cleanup confirmation.
A fine-tuning job checkpoints every ten minutes to a run-specific object prefix; after a simulated interruption, a replacement instance resumes from the last checksum and the original contract is destroyed.
| Failure | Response |
|---|---|
| No compliant offer exists | Pause the run and report the binding constraint; do not silently weaken reliability or price policy. |
| Checkpoint upload fails | Stop training before the recovery window is exceeded and repair storage access. |
| Host goes offline | Use the last external checkpoint on a different host and preserve the affected instance ID for support. |
| Artifact checksum fails | Do not mark the run complete; retain evidence and rerun from the last verified checkpoint. |