NVIDIA A10 at a Glance
Last updated November 3, 2026.
The NVIDIA A10 is a 24GB Ampere GPU for inference and graphics. It draws 150W, fits in one PCIe slot, and moves 600GB/s of memory bandwidth. That is a quarter of an A100 80GB's bandwidth at a fraction of the rental price, which makes it the default card for small models and embeddings.
NVIDIA A10 Specs
| Spec | NVIDIA A10 |
|---|---|
| Architecture | Ampere |
| GPU memory | 24GB GDDR6 |
| Memory bandwidth | 600GB/s |
| FP32 | 31.2 TFLOPS |
| TF32 Tensor | 62.5 TFLOPS (125 with sparsity) |
| FP16 / BF16 Tensor | 125 TFLOPS (250 with sparsity) |
| INT8 Tensor | 250 TOPS (500 with sparsity) |
| INT4 Tensor | 500 TOPS (1,000 with sparsity) |
| RT cores | 72 |
| Interconnect | PCIe Gen4, 64GB/s |
| Form factor | Single-slot, full-height, full-length |
| Max power (TDP) | 150W |
The A10 has no FP8 support. FP8 arrived with Hopper, so FP8 checkpoints run on an A10 only after conversion to INT8, INT4, or FP16.
A10 vs A10G
The A10G is the A10 variant inside AWS EC2 G5 instances. AWS lists 24GB of memory per GPU and up to eight GPUs per instance, and says G5 delivers up to 40% better price performance than the T4-based G4dn. You can buy an A10 from NVIDIA partners; you can only rent an A10G from AWS.
| Instance | A10G GPUs | GPU memory | vCPUs | RAM | Price/hr |
|---|---|---|---|---|---|
| g5.xlarge | 1 | 24GB | 4 | 16GiB | $1.006 |
| g5.2xlarge | 1 | 24GB | 8 | 32GiB | $1.212 |
| g5.12xlarge | 4 | 96GB | 48 | 192GiB | $5.672 |
| g5.48xlarge | 8 | 192GB | 192 | 768GiB | $16.288 |
NVIDIA A10 Price in the Cloud
AWS bills per instance-hour. Modal bills per second while a container runs, which suits bursty inference. The table converts Modal's per-second rates to hourly and places AWS's single-GPU instances next to them.
| GPU | Memory | Modal ($/hr, from per-second rate) | AWS on-demand, 1-GPU instance |
|---|---|---|---|
| NVIDIA T4 | 16GB | $0.59 | $0.526 (g4dn.xlarge) |
| NVIDIA L4 | 24GB | $0.80 | $0.805 (g6.xlarge) |
| NVIDIA A10 / A10G | 24GB | $1.10 | $1.006 (g5.xlarge) |
| NVIDIA L40S | 48GB | $1.95 | n/a |
| NVIDIA A100 40GB | 40GB | $2.10 | n/a |
| NVIDIA A100 80GB | 80GB | $2.50 | n/a |
Which LLMs Fit on an A10 (24GB)
Memory needed is weight bytes plus KV cache. Weight bytes are parameters times bytes per weight: 2 for BF16, 1 for INT8, about 0.5 for 4-bit.
| Model size | BF16 | INT8 | 4-bit | Fits on one A10? |
|---|---|---|---|---|
| 8B | 16GB | 8GB | 4GB | Yes, BF16 with about 8GB left for KV cache |
| 14B | 28GB | 14GB | 7GB | INT8 or 4-bit |
| 32B | 64GB | 32GB | 16GB | 4-bit only, short context |
| 70B | 140GB | 70GB | 35GB | No; needs several GPUs |
Decode speed follows bandwidth. Each generated token reads every weight once, so a single stream on an 8B BF16 model tops out near 600GB/s divided by 16GB, about 37 tokens per second. Batching raises total throughput because many streams share each weight read.
A10 vs T4, L4, and A100
The T4 is the cheaper Turing card with 16GB. The L4 matches the A10's 24GB at a lower hourly rate on both Modal and AWS. The A100 80GB costs about 2.3x as much per hour on Modal and holds 3.3x the memory at 2.04TB/s. For a full A10 vs A100 spec table, see A10 vs A100.
When to Move Past the A10
Move up when the model no longer fits in 24GB at the precision you need, or when single-stream decode near 37 tokens per second is too slow for your users. Hopper and Blackwell cards add FP8 and far more bandwidth. See H200, B200, and GB300 for those tiers, or dedicated inference if you would rather not size GPUs yourself.
Frequently Asked Questions
What is the NVIDIA A10?
The NVIDIA A10 is an Ampere data center GPU with 24GB of GDDR6 memory at 600GB/s, 72 RT cores, and a 150W maximum power draw in a single-slot, full-height PCIe Gen4 card. NVIDIA rates it at 31.2 FP32 TFLOPS and 125 FP16 Tensor TFLOPS (250 with sparsity). It is used for AI inference, virtual workstations, and graphics.
What is the difference between the A10 and the A10G?
The A10G is the A10 variant AWS uses in its EC2 G5 instances. Like the A10, it has 24GB of memory per GPU. AWS offers it in instances with 1, 4, or 8 GPUs, from g5.xlarge at $1.006 per hour on demand in us-east-1. NVIDIA sells the A10; the A10G is available only through AWS.
How much does an NVIDIA A10 cost per hour?
On AWS, g5.xlarge with one A10G costs $1.006 per hour on demand in us-east-1 (Linux), and g5.12xlarge with four A10Gs costs $5.672 per hour. On Modal, the A10 costs $0.000306 per second, about $1.10 per hour, billed only while a container runs.
Can an NVIDIA A10 run Llama 70B?
Not on one card. A 70B model needs about 140GB in BF16 and about 35GB at 4-bit, both above the A10's 24GB. It fits across several GPUs, for example the four A10Gs (96GB total) in an AWS g5.12xlarge with 4-bit or 8-bit weights and tensor parallelism.
A10 vs A100: which should I use?
The A10 suits models up to about 8B in BF16 or 32B at 4-bit, embeddings, and low-traffic endpoints. The A100 80GB has 3.3x the memory and 3.4x the bandwidth (2.04TB/s vs 600GB/s), so it serves larger models and bigger batches with faster decode. On Modal the A10 costs $1.10/hr and the A100 80GB $2.50/hr.
The fastest endpoints are private deployments
Morph's top speeds come from dedicated deployments, not shared public endpoints: speculators trained on your traffic, caching tuned to your workload, and volume discounts over public per-token rates. Over 500 billion tokens per day run this way.
Skip GPU sizing. Call open coding models through one API.
Morph serves GLM-5.3, Kimi K3, and DeepSeek V4 Flash on its own GPU fleet through one OpenAI-compatible API at api.morphllm.com.