GPU cluster cheat sheet: Everything you need to deploy multi-GPU workloads on VOLT

Most developers don't fail at distributed GPU training because they select the wrong model architecture. On the contrary, they misstep when provisioning the wrong cluster and GPU mix, wrong interconnect topology, and wrong scaling strategy. To add insult to injury, they’ll burn $4,000 in three hours trying to figure what the heck went wrong. 

This quick guide exists so you can avoid this mess. 

When we published a GPU cluster quick-reference card on X earlier this quarter, it became one of our highest-bookmarked posts.. We learned that  engineers need a single canonical reference they can pull up mid-deployment instead of another thought-leadership piece about the future of AI infrastructure or meme-posting.

So, we’re giving you what you want: the complete cluster deployment playbook for VOLT, which will guide you from GPU selection to production monitoring. 

Bookmark it! You’ll be back.

GPU quick-reference: H100 vs. A100 vs. L40S

You might assume that choosing the right GPU is all about picking the "best" one. And who could blame you? Picking the best is one of the most basic human impulses. We’re here to tell you that it’s really about matching silicon to your unique workload characteristics. Every GPU in VOLT’s decentralized network has a different price-performance curve, and it all depends on what you’ll actually be doing with it.

When to use which silicon. Let’s break things down a bit more. 

H100

These GPUs are non-negotiable for training runs where collective communication dominates. This means anything involving all-reduce across billions of parameters. With the NVLink 4.0 bandwidth (900 GB/s bidirectional), your gradient synchronization won’t become a bottleneck until you scale past 8 GPUs per node. If you’re training a model from scratch above 30B parameters, start with the H100.

A100

The undisputed workhorse for fine-tuning, LoRA/QLoRA runs, and mid-scale training, for when you require 80 GB of VRAM but don't need the H100’s raw FP16 throughput. Most teams running supervised fine-tuning on models between 7B and 30B parameters will find A100s deliver the best cost-per-token-trained on our network.

L40S 

These cards are the inference play. With 48 GB of GDDR6 and strong single-stream throughput, you’ll find that they serve workloads and real-time generation tasks at a fraction of the cost of the other silicon. So, if you’re deploying a model and not training one, L40S clusters are your best bang for the buck.

Rule of thumb

Train on H100s, fine-tune on A100s, serve on L40S. Deviate only when your profiling data tells you to.

Step-by-step: Deploy your multi-GPU cluster on VOLT

Going from zero to a running distributed workload on VOLT means following a consistent journey. Here’s the path, with annotations that will help you make the decisions that truly matter.

Step 1: Define your cluster spec

Before you touch the console, ask yourself these three questions: 1) How many GPUs do you need? 2) What GPU type matches your workload? and, 3) what’s your interconnect requirement (do your GPUs need to be co-located on the same node, or can they be distributed)?

For training workloads using data parallelism, distributed clusters work fine. For tensor-parallel or pipeline-parallel training, you want GPUs on the same node with NVLink connectivity.

Step 2: Provision through the VOLT console or CLI

Navigate to the deployment interface. Select your GPU type, quantity, and region. A typical configuration might look like this:

YAML

cluster:

  name: llm-finetune-prod

  gpu_type: A100_80GB

  gpu_count: 8

  region_preference: us-west

  networking: private_overlay

  storage: 500GB_nvme

Step 3: Configure networking

Once your GPU is provisioned, VOLT sets up a private overlay network. This is a critical step for multi-node training where NCCL communication needs low-latency paths. Verify connectivity with a quick NCCL all-reduce benchmark before launching your real workload:

Bash

# Verify inter-GPU bandwidth

python -m torch.distributed.run \

  --nproc_per_node=8 \

  --nnodes=1 \

  nccl_benchmark.py --op all_reduce --size 1G

Step 4: Launch your workload

Now with networking validated, it’s time to launch your job. For PyTorch-based workloads, torchrun handles process spawning across nodes. Monitor the first 100 steps closely, looking for GPU utilization dropping below 85% (a sign of communication bottlenecks) or memory usage spiking unexpectedly.

Networking deep dive: Making communication fast

Distributed GPU workloads spend a non-trivial fraction of wall-clock time pushing data around, including: gradient synchronization, activation transfers, and KV-cache sharing. This is your infrastructure, not some secondary set of infra.

Intra-node vs. Inter-node

Within a single node, GPUs communicate over NVLink or PCIe. Across nodes, communication drops to network fabric. On VOLT, that is typically 100 Gbps or higher. You’ll find that the performance gap between intra-node NVLink (900 GB/s on H100s) and inter-node networking (12.5 GB/s on 100 Gbps Ethernet) is roughly 72x.

What’s the practical implication here? Keep tensor-parallel groups within a single node. When it comes to communication across nodes, use data parallelism or pipeline parallelism. Pro-tip: if you try to split a single tensor operation across a network boundary, you’ll spend more time waiting for data than computing.

Cost optimization: More compute, less spend

GPU compute is expensive. Any waste, therefore, is inexcusable.

Right-sizing

Overprovisioning VRAM is one of the most common mistakes that engineers make. You’ll want to run a profiling pass on a single GPU as an initial step. If an A100 80 GB is running at 45 GB peak utilization, you should probably be on L40S 48 GB cards, which will give you what you need at half the cost.

Spot vs. on-demand

VOLT’s decentralized model creates natural price variation. Training runs with checkpointing are ideal candidates for spot-equivalent pricing, which can reduce your training costs by 30-60%. For inference, stay On-Demand. An endpoint that disappears mid-request is an outage not  an optimization.

Avoid "just in case" provisioning

If your training runs on 8 GPUs, don’t provision 16 because you "might scale up." Provision 8 GPUs, validate that this is the right amount for the job, and scale only if profiling demonstrates a clear benefit.

Monitoring: Your observability stack

A distributed GPU cluster that no one on your team monitors will be distributed billing surprise. Nobody wants that. That means that, at minimum, you should be tracking the following metrics:

1. GPU utilization (%): Sustained below 80% means your workload is bottlenecked by CPU preprocessing or data loading.
2. GPU memory usage (GB): Track the high-water mark. If you’re within 2 GB of the limit, one batch-size increase will OOM your job.
3. NCCL throughput: Sudden drops indicate a degraded link or topology change.
4. Cost accumulator: Track cumulative spend in real-time. Set alerts at 50%, 75%, and 90% of your budget.

Do NOT set and forget!

When deploying your multi-GPU clusters, never assume that you can simply"set it and forget it". This type of operation should be understood as  a series of deliberate decisions, with each choice compounding. 

If you get the foundation right, scaling from 8 GPUs to 64 will be a simple configuration shift, and a more pleasant provisioning experience. Get it wrong, and you’ll be debugging networking issues at 2 AM while your cloud bill escalates.

Start deploying: VOLT/docs/clusters. Keep this page open in a tab—it’s the reference you’ll need.