MLU370
The MLU370-S4/S8 is a family of cloud inference accelerators from Cambricon, a Beijing-based AI chip company that has become China's most valuable listed stock (market cap ~$81B as of August 2025).
Publisher review
The MLU370-S4/S8 is a family of cloud inference accelerators from Cambricon, a Beijing-based AI chip company that has become China's most valuable listed stock (market cap ~$81B as of August 2025). Designed for high-density deployment in servers, these half-height, half-length, single-slot cards target applications in smart finance, energy, and manufacturing. The cards draw only 75W (passive cooling) and use the TSMC 7nm process with Cambricon's MLUarch03 architecture. Compared to an equivalently sized GPU, Cambricon claims 3× the video decoding throughput and 1.5× the encoding throughput, making them an option for data centers that need to process many video streams per rack unit.
The MLU370-S4/S8 supports PCIe Gen4 (x16) and packs LPDDR5 memory in 24 GB or 48 GB configurations with a bandwidth of 307.2 GB/s. Peak performance reaches 192 TOPS at INT8, 96 TOPS at INT16, 72 TFLOPS at FP16/BF16, and 18 TFLOPS at FP32. The cards can decode up to 132 HEVC full-HD streams simultaneously, handle 8K video codecs, and process 4000 frames/s for full-HD image decoding and 3000 frames/s for encoding. This makes them suited for workloads like real-time video analytics, AI-powered surveillance, and batch image processing.
Cambricon positions the MLU370-S4/S8 as a domestic alternative to Nvidia's T4 and L4 inference cards, especially in China's push for self-sufficiency. DeepSeek's V3 and V3.1 models have been confirmed compatible with Cambricon chips, giving the cards a software ecosystem advantage in the Chinese market. However, Cambricon's global software stack (MagicMind, NeuWare) is less mature than CUDA, and the company's stock trades at 4,554× historical earnings, reflecting extreme investor speculation rather than current revenue.
The primary trade-offs are software maturity and ecosystem lock-in. While the hardware is competitive on paper (75W TDP, 192 TOPS, 8K decode), developers must port models to Cambricon's proprietary frameworks, which have fewer community resources and pre-trained models than Nvidia's ecosystem. The passive cooling design requires adequate server airflow. Pricing is not publicly listed, and availability outside China is limited. The 75W power cap constrains sustained performance for training workloads, as the card is optimized for inference.
How it works
-
PCIe Gen4 interface
x16 PCIe Gen4 connection provides 16 GT/s per lane, enabling high-bandwidth data transfer between host CPU and accelerator.
-
Multiple precision support
Supports FP32, FP16, BF16, INT16, INT8, and INT4, with peak INT8 performance of 192 TOPS and FP32 at 18 TFLOPS.
-
LPDDR5 memory
24 GB or 48 GB LPDDR5 with 307.2 GB/s bandwidth, balancing capacity and throughput for inference workloads.
-
Video codec engine
Decodes up to 132 HEVC full-HD streams simultaneously, supports 8K resolution, and handles 4000 frames/s image decode.
-
Compact form factor
Half-height, half-length, single-slot card (75W TDP) enables dense server deployment with passive cooling.
-
Cambricon MLUarch03 architecture
7nm TSMC process with custom MLUarch03 core design, optimized for cloud inference and AI acceleration.
-
High-density inference
3× video decode and 1.5× encode throughput compared to same-size GPU, per Cambricon's benchmarks.
Strengths and trade-offs
Strengths
- 75W TDP with passive cooling allows dense deployment of up to 16 cards in a standard 2U server without active fan upgrades.
- 192 TOPS at INT8 delivers competitive inference throughput for batch processing, matching or exceeding Nvidia T4's 130 TOPS INT8 rating.
- 132 HEVC full-HD decode streams per card enable high-channel-count video analytics in surveillance and media applications.
- 48 GB LPDDR5 option provides ample memory for large models like DeepSeek V3, which has been confirmed compatible with Cambricon hardware.
Trade-offs
- Cambricon's MagicMind and NeuWare software stack has a smaller community and fewer pre-trained model repositories than Nvidia's CUDA ecosystem.
- Passive cooling design requires server chassis with strong front-to-back airflow; inadequate airflow can cause thermal throttling under sustained load.
- Pricing is not publicly disclosed, and availability is primarily limited to the Chinese market through partnerships with vendors like Inspur.
- The card is optimized for inference only; its 18 TFLOPS FP32 peak is insufficient for training large models compared to Nvidia A100 (312 TFLOPS FP32).
Pricing context
Not publicly disclosed; available through Cambricon's channel partners (e.g., Inspur) in China, typically as part of server bundles.
Getting started with MLU370
-
Request access from Cambricon
Contact Cambricon or an authorized channel partner like Inspur to request pricing and availability for the MLU370-S4/S8. Provide your server specifications and workload requirements to receive a quote and sample unit.
-
Install the accelerator card
Power down your server and insert the MLU370-S4/S8 into a PCIe Gen4 x16 slot. Ensure adequate front-to-back airflow for passive cooling. Secure the card with the bracket and close the chassis.
-
Set up Cambricon software stack
Download and install the NeuWare SDK and MagicMind framework from Cambricon's developer portal. Follow the installation guide to configure drivers and runtime libraries for your Linux distribution.
-
Port a model to MagicMind
Use MagicMind's model conversion tools to import a trained model from frameworks like PyTorch or TensorFlow. Convert the model to Cambricon's intermediate representation and optimize it for INT8 inference on the MLU370.
-
Run inference and monitor performance
Execute your converted model using the MagicMind runtime API. Monitor card temperature and utilization with Cambricon's system tools. Verify throughput and latency meet your application requirements.
Frequently Asked Questions
What is the Cambricon MLU370-S4/S8 inference card?
The MLU370-S4/S8 is a family of cloud inference accelerators from Cambricon, a Beijing-based AI chip company. These half-height, half-length, single-slot cards are designed for high-density server deployment in applications like smart finance, energy, and manufacturing.
What are the key specifications of the MLU370 inference card?
The MLU370 uses TSMC 7nm process and supports PCIe Gen4 with LPDDR5 memory up to 48 GB. It delivers 192 TOPS at INT8, 18 TFLOPS at FP32, and can decode up to 132 HEVC full-HD streams simultaneously, all within a 75W power envelope.
How does the MLU370 compare to Nvidia's T4 or L4 cards?
Cambricon claims the MLU370 offers 3× video decode and 1.5× encode throughput versus an equivalently sized GPU. It matches or exceeds Nvidia T4's 130 TOPS INT8 rating with 192 TOPS. However, its software ecosystem is less mature than CUDA, limiting global adoption.
What software does the MLU370 use and is it compatible with DeepSeek models?
The MLU370 uses Cambricon's proprietary MagicMind and NeuWare software stack. DeepSeek's V3 and V3.1 models have been confirmed compatible with Cambricon chips, providing a software ecosystem advantage in the Chinese market despite fewer community resources than CUDA.
What are the main use cases for the MLU370 accelerator?
The MLU370 is optimized for cloud inference workloads like real-time video analytics, AI-powered surveillance, and batch image processing. Its ability to decode 132 HEVC streams and handle 8K video makes it ideal for high-channel-count video applications in data centers.
Where can I buy the MLU370 and what is its price?
Pricing for the MLU370 is not publicly disclosed. Availability is primarily limited to the Chinese market through channel partners like Inspur, typically as part of server bundles. It is not widely available outside China.
Alternatives
How MLU370 compares
Direct head-to-head against 3 competitors. Picked by 7wData.
MLU370
- Pricing
- Not publicly disclosed; available through Cambricon's channel partners (e.g., Inspur) in China, typically as part of server bundles.
- Target
- The MLU370-S4/S8 is a family of cloud inference accelerators from Cambricon, a Beijing-based AI chip company that has become China's most valuable listed stock (market
- Strength
- 75W TDP with passive cooling allows dense deployment of up to 16 cards in a standard 2U server without active fan upgrades.
- Watch for
- Cambricon's MagicMind and NeuWare software stack has a smaller community and fewer pre-trained model repositories than Nvidia's CUDA ecosystem.
NVIDIA A100
- Pricing
- Custom/Contact sales; estimated $10,000-$15,000 per card
- Target
- Data center AI training and inference for large-scale workloads
- Deployment
- On-premises, cloud
- Strength
- Mature CUDA ecosystem with broad framework support
- Watch for
- Export restrictions to China; high power consumption (~400W)
Cambricon MLU370
- Pricing
- Custom/Contact sales; estimated $2,000-$4,000 per card
- Target
- AI inference and training in Chinese data centers and edge deployments
- Deployment
- On-premises, edge
- Strength
- Optimized for Chinese AI frameworks and domestic supply chain
- Watch for
- Limited software ecosystem compared to NVIDIA; geopolitical risks
Intel Habana Gaudi 2
- Pricing
- Custom/Contact sales; estimated $8,000-$12,000 per card
- Target
- AI training and inference for enterprise data centers
- Deployment
- On-premises, cloud
- Strength
- Competitive price-performance for large-scale training workloads
- Watch for
- Smaller developer community; integration complexity with existing stacks
User reviews
No user reviews yet. Be the first to write one.
Sources
Reporting on this tool draws on these publicly available sources.