Hardware
Ranked by memory bandwidth, which is what actually governs token generation speed — not teraflops, and not core count.
| Accelerator | Memory○ | Bandwidth○ | Tensor FP16○ | TDP○ | Models fit◐ |
|---|---|---|---|---|---|
| B200 | 192 GB | 8000 | 2250 | 1000 | 1858 |
| H200 SXM | 141 GB | 4800 | 989 | 700 | 1842 |
| H100 SXM 80GB | 80 GB | 3350 | 989 | 700 | 1813 |
| A100 80GB | 80 GB | 2039 | 312 | 400 | 1813 |
| H100 PCIe 80GB | 80 GB | 2000 | 756 | 350 | 1813 |
| RTX PRO 6000 Blackwell Workstation Edition | 96 GB | 1792 | 504 | 600 | 1815 |
| GeForce RTX 5090 | 32 GB | 1792 | 419 | 575 | 1667 |
| GeForce RTX 5090 D | 32 GB | 1792 | 419 | 575 | 1667 |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96 GB | 1792 | 441 | 300 | 1815 |
| A100 40GB | 40 GB | 1555 | 312 | 400 | 1698 |
| GeForce RTX 5090 D V2 | 24 GB | 1344 | 419 | 575 | 1561 |
| RTX PRO 5000 Blackwell | 72 GB | 1344 | 295 | 300 | 1804 |
| RTX PRO 5000 Blackwell | 48 GB | 1344 | 295 | 300 | 1752 |
| GeForce RTX 3090 Ti | 24 GB | 1008 | 160 | 450 | 1561 |
| GeForce RTX 4090 | 24 GB | 1008 | 330 | 450 | 1561 |
| GeForce RTX 4090 D | 24 GB | 1008 | 294 | 425 | 1561 |
| GeForce RTX 5080 | 16 GB | 960 | 225 | 360 | 1187 |
| RTX 6000 Ada Generation | 48 GB | 960 | 364 | 300 | 1752 |
| RTX 5880 Ada Generation | 48 GB | 960 | 277 | 285 | 1752 |
| GeForce RTX 3090 | 24 GB | 936 | 142 | 350 | 1561 |
| GeForce RTX 3080 | 12 GB | 912 | 123 | 350 | 966 |
| GeForce RTX 3080 Ti | 12 GB | 912 | 136 | 350 | 966 |
| Tesla V100 16GB | 16 GB | 900 | 125 | 300 | 1164 |
| Tesla V100 32GB | 32 GB | 900 | 125 | 300 | 1667 |
| GeForce RTX 5070 Ti | 16 GB | 896 | 176 | 300 | 1187 |
| GeForce RTX 5080 Laptop | 16 GB | 896 | — | 150 | 1187 |
| RTX PRO 4500 Blackwell | 32 GB | 896 | — | 200 | 1667 |
| GeForce RTX 5090 Laptop | 24 GB | 896 | — | 150 | 1561 |
| Titan V | 32 GB | 868 | 119 | 250 | 1667 |
| L40S | 48 GB | 864 | 366 | 350 | 1752 |
| RTX A5500 | 24 GB | 768 | 136 | 230 | 1561 |
| RTX A6000 | 48 GB | 768 | 155 | 300 | 1752 |
| RTX A5000 | 24 GB | 768 | 111 | 230 | 1561 |
| GeForce RTX 3080 Ti | 20 GB | 760 | 136 | 350 | 1435 |
| GeForce RTX 3080 | 10 GB | 760 | 119 | 320 | 906 |
| GeForce RTX 4080 Super | 16 GB | 736 | 209 | 320 | 1187 |
| Tesla P100 16GB | 16 GB | 732 | — | 250 | 1164 |
| GeForce RTX 4080 | 16 GB | 717 | 195 | 320 | 1187 |
| A40 | 48 GB | 696 | 150 | 300 | 1752 |
| GeForce RTX 4070 Ti Super | 16 GB | 672 | 176 | 285 | 1187 |
| GeForce RTX 5070 Ti Laptop | 12 GB | 672 | — | 115 | 966 |
| Titan RTX | 24 GB | 672 | 131 | 280 | 1561 |
| RTX PRO 4000 Blackwell | 24 GB | 672 | — | 140 | 1561 |
| GeForce RTX 5070 | 12 GB | 672 | 123 | 250 | 966 |
| Titan V | 12 GB | 651 | 119 | 250 | 966 |
| RTX A4500 | 20 GB | 640 | 95 | 200 | 1433 |
| GeForce RTX 2080 Ti | 11 GB | 616 | 108 | 250 | 954 |
| GeForce RTX 3070 Ti | 8 GB | 608 | 87 | 290 | 624 |
| GeForce RTX 3060 Ti GDDR6X | 8 GB | 608 | 65 | 225 | 624 |
| A10 | 24 GB | 600 | 125 | 150 | 1561 |
| GeForce RTX 4090 Laptop | 16 GB | 576 | — | 150 | 1187 |
| RTX 5000 Ada Generation | 32 GB | 576 | 261 | 250 | 1667 |
| Titan Xp | 12 GB | 548 | — | 250 | 966 |
| GeForce RTX 3080 Ti Laptop | 16 GB | 512 | — | 150 | 1187 |
| GeForce RTX 4070 Ti | 12 GB | 504 | 160 | 285 | 966 |
| GeForce RTX 4070 | 12 GB | 504 | 117 | 200 | 966 |
| GeForce RTX 4070 Super | 12 GB | 504 | 142 | 220 | 966 |
| GeForce RTX 2080 SUPER | 8 GB | 496 | 89 | 250 | 624 |
| GeForce GTX 1080 Ti | 11 GB | 484 | — | 250 | 954 |
| GeForce RTX 4070 GDDR6 | 12 GB | 480 | 117 | 200 | 966 |
| GeForce RTX 2070 SUPER | 8 GB | 448 | 73 | 215 | 624 |
| GeForce RTX 3080 Laptop | 16 GB | 448 | — | 150 | 1187 |
| RTX A4000 | 16 GB | 448 | 77 | 140 | 1164 |
| GeForce RTX 5060 Ti | 8 GB | 448 | 95 | 180 | 624 |
| GeForce RTX 5060 Ti | 16 GB | 448 | 95 | 180 | 1187 |
| GeForce RTX 3070 | 8 GB | 448 | 81 | 220 | 624 |
| GeForce RTX 5060 | 8 GB | 448 | 77 | 145 | 624 |
| GeForce RTX 3060 Ti | 8 GB | 448 | 65 | 200 | 624 |
| GeForce RTX 2060 SUPER | 8 GB | 448 | 57 | 175 | 624 |
| GeForce RTX 2070 | 8 GB | 448 | 60 | 175 | 624 |
| GeForce RTX 2080 | 8 GB | 448 | 81 | 215 | 624 |
| GeForce RTX 4080 Laptop | 12 GB | 432 | — | 150 | 966 |
| RTX 4500 Ada Generation | 24 GB | 432 | 159 | 210 | 1561 |
| GeForce RTX 5060 Laptop | 8 GB | 384 | — | 100 | 624 |
| GeForce RTX 5070 Laptop | 8 GB | 384 | — | 100 | 624 |
| GeForce RTX 5050 Laptop | 8 GB | 384 | — | 100 | 624 |
| RTX 4000 Ada Generation | 20 GB | 360 | 107 | 130 | 1433 |
| GeForce RTX 3060 | 12 GB | 360 | 51 | 170 | 966 |
| Tesla P40 | 24 GB | 346 | — | 250 | 1561 |
| GeForce RTX 2060 | 12 GB | 336 | 57 | 184 | 966 |
| GeForce RTX 2060 | 6 GB | 336 | 52 | 160 | 370 |
| GeForce RTX 3060 OEM | 6 GB | 336 | 57 | 185 | 370 |
| GeForce RTX 5050 | 8 GB | 320 | 53 | 130 | 624 |
| Tesla T4 | 16 GB | 320 | 65 | 70 | 1164 |
| L4 | 24 GB | 300 | 121 | 72 | 1561 |
| RTX PRO 2000 Blackwell | 16 GB | 288 | — | 70 | 1164 |
| RTX A2000 | 12 GB | 288 | 32 | 70 | 965 |
| RTX A2000 | 6 GB | 288 | 32 | 70 | 364 |
| GeForce RTX 4060 Ti | 8 GB | 288 | 88 | 160 | 624 |
| GeForce RTX 4060 Ti | 16 GB | 288 | 88 | 165 | 1187 |
| RTX 4000 SFF Ada Generation | 20 GB | 280 | 77 | 70 | 1433 |
| NVIDIA DGX Spark (GB10)unified | 128 GB | 273 | 125 | 140 | 1840 |
| GeForce RTX 4060 | 8 GB | 272 | 60 | 115 | 624 |
| GeForce RTX 3060 | 8 GB | 240 | 51 | 195 | 624 |
| GeForce RTX 3050 | 8 GB | 224 | 36 | 130 | 624 |
| GeForce RTX 3050 OEM | 8 GB | 224 | 32 | 130 | 624 |
| RTX 2000 Ada Generation | 16 GB | 224 | 48 | 70 | 1164 |
| RTX A1000 | 8 GB | 192 | 27 | 50 | 579 |
| GeForce RTX 3050 | 6 GB | 168 | 27 | 70 | 370 |
| RTX A400 | 4 GB | 96 | 11 | 50 | 224 |