LLM fine-tuning budget guide: GPU costs, timelines, and what to Spend

TL;DR

• A LoRA fine-tune on a 7B model costs under $10. 
• A 70B QLoRA run costs $15–30. 
• Full fine-tuning a 70B on 8 GPUs for a day costs $200–300. 
• If your actual spend is materially higher, the gap is almost certainly the GPU pricing layer, not the job itself.

Fine-tuning a large language model costs anywhere from $3 to $3,000. Model size, GPU tier, and whether you're running LoRA adapters or attempting a full-weight update are all factors that can impact pricing. The reality is that most teams overspend. They don’t do this because fine-tuning is expensive in and of itself, but because they default to on-demand hyperscaler pricing for jobs that could run on spot GPU cloud at 60–70% less. 

This LLM Fine-Tuning Budget Guide breaks down the actual math for AI teams. You’ll get a practical foundation in what each combination of model size, GPU, and technique costs in 2026. We’ll also share what the real budget thresholds are, and how your AI team can stop paying AWS rates for work that doesn't need AWS infrastructure.

What your fine-tuning job actually consumes

Before making your move on the GPU, think about what the job is actually doing. Fine-tuning is memory-bound before it is compute-bound. So, the dominant cost driver is getting gradients, optimizer states, and activations to fit in VRAM not raw FLOPS.

A 7B parameter model in BF16 occupies roughly 14 GB for weights alone. Add optimizer states (AdamW doubles that to ~28 GB), activations (batch-size-dependent, but typically 4–8 GB per layer at batch size 1), and you're sitting at 35–50 GB for a full fine-tune. That pushes you onto an A100 80GB or larger.

LoRA and QLoRA shifts the math entirely. LoRA freezes base weights and trains only low-rank adapter matrices, typically to the tune of 1–10% of parameters. QLoRA adds 4-bit quantization of the frozen base, dropping VRAM requirements by roughly 60%. A 7B QLoRA fine-tune fits comfortably on a single A100 40GB (40 GB) or even a high-memory consumer card. A 70B QLoRA fits on one A100 80GB.

Here’s a quick view of practical VRAM floor by model size and technique. 

These numbers are practical estimates at batch size 4. Larger batches increase activation memory linearly.

GPU tiers and what they cost in 2026

In the last few years the GPU market has stratified into basically  three budget bands. The band you pick should flow directly from the VRAM requirement above, not from brand loyalty or what your hyperscaler makes easy.

Budget tier: A100 40GB and RTX 4090

The A100 40GB at $1.20/hr on VOLT spot is the right GPU for any job that fits in 40 GB. That is, a 7B full fine-tune, 13B LoRA, 30B or 70B QLoRA. A100 40GB delivers 1.6 TB/s memory bandwidth, making it fast enough that most 7B fine-tunes wrap up in under 2 hours. RTX 4090 (24 GB consumer card) works for very small jobs or quantized inference but lacks ECC memory and NVLink support. We don’t recommend this option for multi-day production runs.

Mid tier: A100 80GB and H100 PCIe

At identical VOLT spot pricing ($1.49/hr for both), the H100 PCIe is a straightforward upgrade over the A100 80GB. Your team will get the same price, same 80 GB VRAM, and 67% more memory bandwidth (3.35 TB/s vs. 2.0 TB/s). For 13B–70B LoRA jobs, the H100 PCIe cuts wall-clock time by 30-40% at zero cost premium. We recommend defaulting to H100 PCIe unless there are availability constraints.

High tier: H100 SXM and H200 SXM

H100 SXM ($2.20/hr on VOLT) adds NVLink interconnect, which matters for multi-GPU jobs where inter-GPU bandwidth bottlenecks training. For single-GPU work, it's very rarely worth spending the 48% premium over H100 PCIe. H200 SXM ($3.50/hr) delivers 141 GB VRAM and 4.8 TB/s bandwidth, which makes it an ideal fit for 70B+ full fine-tuning or multi-epoch runs on very large datasets where the extra memory eliminates gradient accumulation overhead.

Worked cost scenarios by fine-tuning workload type

The scenarios below use VOLT spot pricing as a baseline, with AWS on-demand figures included for comparison purposes.

Scenario 1: Instruction-tuning a 7B model (LoRA, 10K samples)

A typical dataset for instruction-tuning a 7B model (10,000 examples, average 512 tokens) takes roughly 1-1.5 hours on a single A100 40GB with LoRA at batch size 8.

• A100 40GB on VOLT spot: 1.25 hrs × $1.20 = $1.50
• A100 80GB on AWS (p3.2xlarge equivalent): ~$3.06/hr × 1.25 = $3.83
• H100 PCIe on VOLT: 0.75 hrs × $1.49 = $1.12

Let’s say a team needs to run this job 10 times during experimentation, that works out to $11.20 on VOLT H100 PCIe vs. $38.30 on AWS. As you see, the experimentation cycle is where budget gets burned.

Scenario 2: Domain-adapting a 13B model (QLoRA, 50K samples)

A 50,000-example domain adaptation run (medical notes, legal documents, internal documentation) on a 13B model with QLoRA takes roughly 6-8 hours on a single A100 80GB.

• A100 80GB on VOLT spot: 7 hrs × $1.49 = $10.43
• H100 PCIe on VOLT spot: 4.5 hrs × $1.49 = $6.71
• AWS p4d.24xlarge (minimum 8 GPUs, billed as node): $32.77/hr × 7 hrs = $229.39 — for eight A100s when you only needed one

Let’s be real: the AWS minimum node size is the single biggest source of waste for teams with single-GPU workloads. You’ll pay for seven GPUs that you won’t use. You can be smarter with your budget. 

Scenario 3: Full fine-tuning a 70B model (BF16, 8× GPU, 24 hours)

Full BF16 fine-tuning of a 70B model requires at minimum 8× A100 80GB or 4× H200 SXM to hold weights + optimizer states. For a 24-hour run:

The H100 SXM cluster wins on total cost for 70B full fine-tuning because it completes the equivalent job in fewer hours. NVLink bandwidth means gradient synchronization isn't a bottleneck. For teams where 24-hour wall-clock is acceptable, 8× A100 on VOLT spot ($286) is the budget option.

Your fine-tuning decision framework: matching budget to technique

Most teams approach fine-tuning by asking the wrong question: “which GPU should I use?" What you should really be asking is "what does my job actually need?" With that in mind, here’s your decision path:

Step 1: Can you use QLoRA?

If you're fine-tuning for task adaptation, domain knowledge injection, or instruction following, and you're not trying to change the model's fundamental reasoning patterns, QLoRA will get you 90%+ of the result at 30–50% of the VRAM cost. Start here unless you have a real documented reason not to do so.

Step 2: How large is your model?

Jobs under 13B on QLoRA fit on a single A100 40GB ($1.20/hr). Jobs up to 70B fit on a single A100 80GB or H100 PCIe ($1.49/hr). Jobs on 70B full fine-tune or 405B anything require multi-GPU clusters and dedicated reservation.

Step 3: How many training runs will you do?

Let’s say you're iterating on hyperparameters, dataset curation, or prompt format, which means 10–20 training runs before you converge on a final models. Well, the per-run cost will multiply, so that a $5 experiment run repeated 15 times ends up being $75, while a $50 experiment run repeated 15 times leads to $750. Think in cheap-per-run instead of cheap-per-hour.

Step 4: What's your tolerance for preemption?

Spot GPU instances can be preempted. For jobs clocking in under 4 hours, the statistical risk is low enough that spot is almost always the right call. For multi-day jobs on 70B+ models, checkpoint every 500-1,000 steps using Hugging Face Trainer or Axolotl, or reserve a dedicated cluster.

Why VOLT for fine-tuning

VOLT is a decentralized GPU marketplace that aggregates enterprise-grade hardware. We offer H100 SXM, A100 80GB, and H200 from data centers and verified suppliers across the globe. For fine-tuning workloads, three major factors make VOLT’s market the right default choice over hyperscalers. 

Single-GPU access

If you go with AWS, they will force you to rent full p4d.24xlarge nodes (8× A100). For the majority of fine-tuning jobs that need one or two GPUs, that’s overkill. Why? You're paying for six idle GPUs. With VOLT, you rent precisely the number of GPUs to complete your fine-tuning job.

Flat, transparent pricing

VOLT pricing is published, transparent, and consistent. We offer an A100 40GB at $1.20/hr spot, an A100 80GB and H100 PCIe at $1.49/hr spot, and H100 SXM at $2.20/hr, H200 at ~$3.50/hr. Even better, there will be no egress fees, no storage volumes that outlive your cluster, and no support tier percentages applied to your monthly total.

NVLink cluster availability

For multi-GPU fine-tuning jobs where inter-GPU bandwidth determines training speed, VOLT offers curated NVLink H100 clusters. You specify node count, GPU type, and interconnect requirement, and our marketplace surfaces matching options for; a feature that is notably absent from providers that only offer shared-tenant multi-node configurations.

A quick practical note

VOLT's deployment interface supports Docker-compatible containers. Bring a Hugging Face Transformers + PEFT image, configure your fine-tuning script, and launch. Most teams are running their first fine-tuning job within 20 minutes of account creation.

For teams ready to move beyond experimentation into regular production fine-tuning (monthly model refreshes, domain adapters for new document sets, RLHF iterations), VOLT's reserved capacity pricing runs 15-20% below spot rates on 30-day commitments.

Related Questions

Q: How much does it cost to fine-tune GPT-4-class models versus open-source LLMs?

GPT-4-class fine-tuning through OpenAI's API is priced per token: as of 2026, GPT-4o fine-tuning runs approximately $25/million training tokens. A 50K-example dataset at 512 tokens average = 25.6M tokens = ~$640 per fine-tuning run. For comparison, fine-tuning Llama 3.1 70B (a comparable-tier open-source model) with QLoRA on VOLT costs ~$26 for the same run, with full weight ownership and no API dependency. The open-source route is roughly 25× cheaper at this scale.

Q: What is the minimum GPU to fine-tune Llama 3.1 8B?

Llama 3.1 8B with QLoRA fits on a 16 GB GPU — an A4000 or consumer card. On VOLT, an A100 40GB at $1.20/hr handles it with headroom for larger batch sizes, completing a 10K-sample instruction tune in under 2 hours ($2.40 total). Full fine-tuning Llama 3.1 8B in BF16 requires ~50 GB VRAM — a single A100 80GB handles it at $1.49/hr.

Q: How do I reduce fine-tuning costs without sacrificing model quality?

Three levers in order of impact: (1) Use QLoRA instead of full fine-tuning — it cuts VRAM by ~60% with minimal quality loss on adaptation tasks. (2) Run on spot GPU cloud rather than on-demand hyperscaler — VOLT spot vs. AWS on-demand is typically a 60–70% cost reduction. (3) Optimize your dataset curation before training — a well-curated 5K-example dataset often outperforms a noisy 50K-example set, cutting both job duration and iteration costs.

Q: Is fine-tuning cheaper than prompt engineering or RAG for most use cases?

It depends on query volume and task type. For tasks requiring consistent output format, specialized vocabulary, or domain-specific reasoning that doesn't live in retrievable documents, fine-tuning is more cost-effective at scale — one $15 fine-tuning run amortized over 10M queries is effectively free. For knowledge injection or tasks where the source documents change frequently, RAG is cheaper to maintain. Most production systems end up combining both: a fine-tuned base for style and format, RAG for current factual content.

Q: How many training steps does a typical fine-tuning job require?

For LoRA and QLoRA instruction-tuning on 7B–70B models, most practitioners converge in 1,000–3,000 steps. At a batch size of 8 and 512-token sequences, 10,000 examples completes in roughly 1,250 steps per epoch — one to two epochs is typically sufficient. Overfitting past 3 epochs on small instruction datasets is common. Monitor validation loss and stop early; most cost waste in fine-tuning is from running too long, not from GPU selection.