AI & GPU compute
Run AI inference with vLLM or Ollama, fine-tune with QLoRA or render with Blender on a dedicated GPU server whose 20 GB of ECC VRAM no other tenant shares.
No vGPU partitioning and no neighbors: all 20 GB of VRAM and every Tensor core of the RTX 4000 SFF Ada work for you alone.
Weights, KV cache and batch size all live in VRAM, so size the card for your model and context first, then think about speed.
A GPU that serves or trains around the clock costs the same every month, with no hourly meter running through idle weekends.
Weights, prompts and datasets sit on your own NVMe behind your own firewall, with full root and IPMI access.
Recommended servers
From a first deployment to a busy production setup. Live stock and prices; in-stock picks are online in minutes.
CPU inference and RAG
16C / 32T · 4.3–5.7 GHz
No GPU, but 16 Zen 5 cores and 128 GB DDR5 run embeddings, vector search and small quantized models with llama.cpp next to your app.
Our pick7–14B models, SDXL, rendering
20 GB VRAM · EPYC 4344P · 8C / 16T · 3.8–5.3 GHz
20 GB ECC VRAM, 192 Tensor cores and 48 RT cores: quantized 7–14B LLMs, QLoRA tuning, SDXL or Blender OptiX on one in-stock card.
30B-class models
48 GB VRAM · EPYC 9254 · 24C / 48T · 2.9–4.15 GHz
48 GB of VRAM for ~30B models in 4-bit or long contexts. Sold out for now: start on the RTX 4000 SFF Ada, or ask for a custom quote.
Sizing
These are estimates, not measurements: long contexts and many parallel requests grow the KV cache well beyond these figures, so check your framework’s memory report.
| Workload | Rule-of-thumb VRAM | Suggested plan |
|---|---|---|
| Embeddings, rerankers, small models on CPU | No GPU; system RAM for model and vector index | AMD Ryzen 9 9950X |
| 7–8B LLM in 4-bit or 8-bit for chat or RAG | ~5–9 GB of weights plus KV cache: fits 20 GB | NVIDIA RTX 4000 SFF Ada |
| 13–14B LLM in 4-bit | ~8–9 GB of weights, room for moderate context | NVIDIA RTX 4000 SFF Ada |
| QLoRA fine-tune of a 7–8B model | Fits 20 GB with short sequences, small batches | NVIDIA RTX 4000 SFF Ada |
| SDXL or ComfyUI pipelines, Blender Cycles | SDXL fits easily; larger models in FP8 | NVIDIA RTX 4000 SFF Ada |
| ~30B-class LLM in 4-bit, or long contexts | ~17–20 GB of weights; 48 GB leaves KV room | NVIDIA L40S |
Expert notes
Short, practical advice from our engineers for this workload.
Three things compete for the 20 GB on the RTX 4000 SFF Ada:
In vLLM, cap --max-model-len at what your app actually sends and tune --gpu-memory-utilization. In llama.cpp, -ngl sets how many layers go to the GPU; the rest run from system RAM, slower but fine for testing a model that doesn’t quite fit.
Install Ubuntu 24.04 or 26.04 LTS in one click, then let Ubuntu pick a tested server driver:
ubuntu-drivers list --gpgpu shows the recommended branch and ubuntu-drivers install --gpgpu installs it. Add the matching nvidia-utils-…-server package if nvidia-smi is missing, then reboot.nvidia-smi should list the card and its 20 GB; nvidia-smi -q -d ECC shows the ECC mode.nvidia-ctk runtime configure --runtime=docker and restart Docker.The host only needs the driver. PyTorch wheels and the official vLLM images bring their own CUDA runtime, so projects can use different CUDA versions side by side as long as the driver is recent enough. Pin the driver package so an unattended upgrade doesn’t swap it under a running job.
vLLM, llama.cpp’s llama-server and Ollama all speak an OpenAI-compatible HTTP API. That makes swapping backends easy, and exposing one to the whole internet just as easy.
127.0.0.1, as Ollama does by default on port 11434, and publish it through a reverse proxy with TLS and authentication.llama-server accept --api-key; treat it as a minimum, not as access control.Root access means you install exactly what you need. These are proven choices for this workload.
FAQ
As a rule of thumb, 7–8B models fit comfortably in 4-bit or 8-bit with room for context, and 13–14B models fit in 4-bit. A 7–8B model in FP16 loads too but leaves little space for the KV cache. ~30B-class models in 4-bit need roughly 17–20 GB for the weights alone, so they belong on a 48 GB card. Check your framework’s memory estimate before committing.
Yes, with parameter-efficient methods. QLoRA keeps the base model in 4-bit and trains small adapter matrices, so a 7–8B model fits in 20 GB with modest sequence lengths and batch sizes, and gradient checkpointing stretches that further. Unsloth, Axolotl and Hugging Face PEFT handle the setup. Full fine-tuning of the same model needs several times more memory than one card offers.
Start on the in-stock RTX 4000 SFF Ada if your model fits in 20 GB, or quantize one step further while you wait: popular configurations are restocked regularly and you can ask to be notified. If you need 48 GB or several GPUs now, request a custom build. Quotes arrive within one business day, and delivery typically takes 2–5 business days.
Yes. SDXL runs comfortably in 20 GB, and ComfyUI installs like any Python app or container. Newer, larger image and video models often need FP8 or GGUF-quantized checkpoints to fit, and batch size is the first thing to lower when memory runs out. Keep the interface behind authentication or a VPN, because custom nodes are arbitrary Python code.
Hourly GPUs suit short experiments. Once a model serves users around the clock or a training queue never empties, a monthly server gives a fixed bill, a GPU nobody else shares, local NVMe for weights and datasets, and no egress meter on an unmetered port. If your GPU would sit idle most of the month, hourly capacity is probably the better deal.
Pick one of the recommended in-stock servers and it can be online in minutes. Need something bigger or different? Our engineers quote custom builds within one business day.