I Helped a Friend Price Out DeepSeek-R1 671B. The Numbers Were Brutal.

I recently helped a friend in financial risk management evaluate a private deployment plan for DeepSeek-R1 671B. After crunching the numbers, I was a bit stunned myself — this is genuinely not something a small company can casually take on. A bit of background first: DeepSeek's inference side currently runs 13,000+ NVIDIA GPUs (H100 + A100) and can only support 5 million DAU, while a peak of 40 million DAU, as assessed, requires a 7× expansion of current compute capacity. The company hasn't announced any consumer-side scaling plans for now, and my read is that they'll most likely shift resources toward the B2B side going forward.

How to Set Up the Hardware

At the 671B scale, the appetite for VRAM and compute is enormous. After comparing options, there are basically three viable configurations:

• 32 NVIDIA A100 (40GB) GPUs
• 16 NVIDIA H100 or H800 (80GB) GPUs
• 8 NVIDIA H200 (141GB) GPUs

All three paths share the same prerequisite: INT8 quantization is mandatory. In my own testing, without quantization, FP16 (16-bit floating point) requires 1.342 TB of VRAM, and FP32 (32-bit floating point) shoots straight up to 2.684 TB — there is simply no way to fit that on any single GPU. Quantization compresses parameters from FP16 or FP32 down to INT4, slashing both VRAM and compute requirements by a significant margin. This is an unavoidable step when running ultra-large-scale models.

Which GPU Offers the Best Value

My first reaction after looking at this table: the H200 leads across the board in VRAM capacity, compute performance, and memory bandwidth compared to both H100/H800 and A100. If the budget allows, the 8× H200 configuration uses the fewest cards, puts the least load on each GPU, and is the most hassle-free from an engineering standpoint. But "budget allows" is itself a luxury in the 2025 GPU market.

Where the Money Comes From: Cloud vs. Self-Purchase

Going with AWS Cloud

I had my friend run a first round of estimates using the cloud approach. The pricing difference between the Ningxia and US East regions is quite significant:

Ningxia Region

US East Region

There's a gotcha I've hit before: the instance types listed above are only available to **new accounts**. Existing accounts need to file a support case to request access, and the resource pool is limited, so waiting in a queue is the norm. So if your company already has a batch of long-standing AWS accounts, this path may not work out — you'll need to confirm in advance.

Buying Your Own Hardware

The other option is to buy GPUs outright and build your own data center. I broke the costs down into four components:

• **GPUs**: The largest share. At current market prices (the catch is that having the money doesn't guarantee you can actually buy them), a single A100 (80GB) runs about 100,000 RMB, a single H100/H800 (80GB) about 200,000 RMB, and a single H200 (141GB) about 400,000 RMB.
• **Servers**: CPUs, memory, storage, and other supporting components — a high-performance server runs roughly 500,000 RMB.
• **Power**: Running a model at this scale means real, tangible electricity costs. Don't overlook this.
• **Operations**: Staff salaries, equipment maintenance — ongoing long-term expenses.

All told, the total investment for a private deployment of DeepSeek-R1 671B lands somewhere in the range of 3 to 5 million RMB. If you go with the 32× A100 configuration, the GPUs alone cost 3.2 million RMB, and once you add servers and the data center facility, 5 million RMB is the floor.

Optimization Techniques I've Actually Used

After deployment, if you want to squeeze costs further and boost throughput, here are the techniques I pretty much always apply:

• **Model parallelism**: Spread parameters across multiple GPUs to immediately reduce per-GPU VRAM pressure.
• **Tensor parallelism**: Split matrix operations across multiple GPUs to lighten the compute load on each one.
• **Pipeline parallelism**: Divide the model into several stages, run each stage on different GPUs, and push overall throughput up.
• **Mixed precision**: Use FP16 / INT8 during training or inference to reduce both VRAM and compute overhead simultaneously.

These four techniques are not mutually exclusive. I typically stack model parallelism with tensor parallelism, and that combination yields the most noticeable results.

Easy-to-Overlook Pitfalls

• **Software stack compatibility**: Framework, driver, and communication library versions need to match the 671B inference engine. I once had multi-GPU communication completely hang because of a minor NCCL version mismatch, and it took me half a day to track down.
• **Cooling**: When 32 A100s are running at full tilt, the data center temperature can spike above 35°C. If your cooling solution isn't up to par, thermal throttling is just a matter of time.
• **Inter-GPU networking**: Whether GPUs communicate over NVLink or InfiniBand makes a several-fold difference in bandwidth and directly determines inference latency.

My Take

Private deployment of the 671B model isn't a "buy a few GPUs" problem — it's an entire engineering project. Which hardware path to choose and how many optimizations to apply depends on your own QPS requirements, budget ceiling, and ops team's capabilities. If your DAU hasn't reached the million level yet, I'd lean toward starting with the cloud and paying as you go, then considering self-purchase once volume scales up — once you've sunk 5 million RMB, the payback period is hard to pin down.