Foreword
DeepSeek, the open-source AI model from China, has taken the developer community by storm. What makes it stand out is not just the performance, but the price. Training costs that once ran into hundreds of millions of dollars were achieved at a fraction of that budget, and the released models are free to download, modify, and deploy. For teams that want local AI inference without paying per-token API fees, DeepSeek is one of the most cost-effective options available.
But cheap to download does not mean cheap to run. The same DeepSeek family spans model sizes from 1.5B parameters to 671B parameters, and the hardware gap between them is enormous. A 1.5B model runs comfortably on a $5 VPS. The full 671B model needs a multi-GPU cluster with hundreds of gigabytes of VRAM. Picking the wrong server for your target model means either wasting money on hardware you do not need, or hitting out-of-memory errors on hardware that cannot keep up.
This guide breaks down server requirements by model size, from entry-level to full bare metal GPU deployments. It covers GPU selection, system memory, storage, network bandwidth, and quantization trade-offs, so you can match hardware to the DeepSeek variant you actually plan to run.
Model sizes and their real requirements
DeepSeek offers a range of model sizes, and each tier has different hardware demands. The list below summarizes the practical requirements for the most commonly deployed variants.
DeepSeek-R1-Distill-Qwen-1.5B: Runs on CPU or a small GPU. 2 cores and 4GB RAM are enough for basic testing with quantized weights.
DeepSeek-R1-Distill-Qwen-7B: Needs 4 cores and 8GB RAM for CPU inference, or a GPU with 8GB+ VRAM for acceptable speed.
DeepSeek-R1-Distill-Qwen-14B: FP16 requires roughly 28GB VRAM. A single RTX 4090 24GB can run it with INT8 quantization.
DeepSeek-R1-Distill-Qwen-32B: FP16 needs around 64GB VRAM. Two RTX 4090s or one A100 80GB handles it comfortably.
DeepSeek-R1-70B: Requires 2×V100 32GB at minimum for FP16, or 4×RTX 4090 with quantization.
DeepSeek-V3 / R1 671B: FP8 needs approximately 671GB VRAM. This is a multi-node, multi-GPU deployment territory, typically 8×H100 or more.
For most individual developers and small teams, the 7B and 14B variants offer the best balance between capability and cost. The 32B model is the upper limit for single-server deployments with consumer-grade hardware. For anything above 32B, bare metal GPU servers become the practical choice.
GPU selection by model tier
The GPU is the single most important component for DeepSeek inference performance.
Entry-level (1.5B / 7B): No dedicated GPU required. CPU inference works for testing. If you want GPU acceleration, an RTX 3060 12GB or RTX 4060 Ti 16GB is sufficient.
Mid-range (14B / 32B): RTX 4090 24GB is the most cost-effective choice for 14B. For 32B, two RTX 4090s or a single A100 40GB/80GB.
Professional (70B): 2×V100 32GB, 2×A100 40GB, or 4×RTX 4090 with INT4 quantization. This tier benefits from bare metal GPU servers because multi-GPU communication over PCIe or NVLink performs better without virtualization overhead.
Enterprise (671B): 8×H100 80GB or equivalent, often across multiple nodes with high-speed interconnect. Bare metal GPU clusters are the standard deployment model here, since virtualized environments add latency and limit direct hardware access.
For teams renting GPU compute rather than buying hardware, the key decision is VRAM capacity and memory bandwidth. DeepSeek inference is memory-bandwidth-bound, not compute-bound, so a GPU with higher memory bandwidth (like the H100’s HBM3) will outperform a GPU with more CUDA cores but slower memory.
Why bare metal matters for DeepSeek
For DeepSeek deployments above the 14B tier, bare metal GPU servers offer advantages that shared or virtualized environments cannot match.
Full GPU access: No virtualization layer between the model and the hardware. CUDA, cuDNN, and NCCL run at native performance, which matters for multi-GPU tensor parallelism.
Dedicated VRAM: Your model gets the full VRAM of every card. No neighbor workloads competing for memory bandwidth.
Stable P99 latency: Inference latency does not spike because another tenant started a training job on the same physical host. For production API serving, predictable latency is critical.
Multi-GPU interconnect: Bare metal allows direct NVLink or PCIe Gen5 communication between GPUs, which is essential for tensor parallelism across 2, 4, or 8 cards.
Data isolation: Models and inference data stay on physically isolated hardware, which simplifies compliance and audit requirements.
No noisy neighbor on storage: NVMe drives are dedicated, so model loading and KV cache writes are not affected by other tenants.
For teams deploying 32B, 70B, or 671B models, a bare metal GPU server with 2 to 8 GPUs is often the difference between a deployment that works and one that keeps hitting memory or latency walls.
System memory and storage
System RAM matters for model loading and KV cache management.
System RAM: At least equal to model size in FP16 for CPU offloading. For 7B models, 16GB is comfortable. For 32B, 64GB+ recommended. For 70B, 128GB+. Bare metal servers can be configured with 256GB or more, which helps when offloading larger models.
Storage: NVMe SSD is strongly recommended. Model files for 7B are around 15GB, 32B around 65GB, and 70B around 140GB. Loading from SATA SSD adds noticeable startup delay. Bare metal configurations typically ship with NVMe RAID, which improves read speed and redundancy.
Storage speed: Sequential read of at least 2GB/s for large models. NVMe Gen4 delivers 5GB/s+, which cuts model load time significantly.
Disk space: Allocate 2-3x the model file size for quantized versions, cached weights, and logs.
Network and bandwidth considerations
For teams serving DeepSeek inference to users, network becomes a factor.
API serving: If you expose DeepSeek through an API, bandwidth requirements depend on concurrent users and response length. Text responses are small, so 10-50Mbps is usually sufficient for moderate traffic.
Model distribution: If you distribute model weights to multiple nodes, high bandwidth helps. A 140GB model file transferred at 1Gbps takes about 20 minutes.
Unlimited bandwidth plans: For inference services with unpredictable traffic, unlimited bandwidth VPS or bare metal servers avoid overage charges. A Los Angeles node with CN2 GIA routing can serve both North American and Asian users with acceptable latency.
Latency: For interactive chat applications, total round-trip latency matters. Co-locating the GPU server near your users reduces perceived delay.
Quantization trade-offs
Quantization reduces VRAM requirements at the cost of some accuracy.
FP16: Full precision, highest quality, highest VRAM. 7B needs 14GB, 32B needs 64GB.
INT8: Roughly half the VRAM of FP16, minimal quality loss. 7B needs 7GB, 32B needs 32GB.
INT4: Quarter of FP16 VRAM, noticeable but often acceptable quality loss. 7B needs 4GB, 32B needs 16GB, 70B needs 35GB.
GPTQ / AWQ / GGUF: Different quantization formats with varying compatibility. GGUF works well with Ollama and llama.cpp. GPTQ and AWQ are common for vLLM.
For a 14B model on a single RTX 4090 24GB, INT8 quantization is the sweet spot. For 32B on the same card, INT4 is necessary. For 70B and above, bare metal GPU servers with 4 or 8 cards allow running FP16 or INT8 without aggressive quantization.
A practical selection framework
To choose a server for DeepSeek deployment, work through these steps:
Pick the model size based on your quality and capability requirements.
Determine the quantization level based on acceptable quality loss.
Calculate VRAM requirement: model size × bytes per parameter × quantization factor.
Choose a GPU or GPU combination that meets the VRAM requirement with headroom for KV cache.
For 32B and above, evaluate bare metal GPU servers to avoid virtualization overhead and get dedicated multi-GPU interconnect.
Size system RAM at 1-2x the model size for CPU offloading.
Choose NVMe storage with at least 2GB/s sequential read.
Estimate bandwidth needs based on concurrent users and response size.
For most teams starting with DeepSeek, a 14B model on a single RTX 4090 with INT8 quantization, 32GB system RAM, and 1TB NVMe storage is a solid starting point. For teams moving to 32B or 70B production inference, a bare metal GPU server with 2×A100 80GB or 4×RTX 4090 is the next step up.
Summary
DeepSeek deployment server selection comes down to matching model size, quantization level, and hardware capacity. The 1.5B and 7B models run on modest VPS instances. The 14B and 32B models need dedicated GPUs with 24GB to 64GB VRAM. The 70B and 671B models require bare metal GPU servers with multi-GPU or multi-node clusters.
For teams that want to avoid hardware procurement and depreciation, renting GPU compute or bare metal GPU servers with the right VRAM and memory bandwidth is often more practical than buying. A Los Angeles bare metal node with high-bandwidth memory and unlimited bandwidth connectivity covers most DeepSeek deployment scenarios for both North American and Asian users.
Thank you very much for reading to the end!
