DeepSeek: Chinese AI that is shaking up the rules of the game

There’s been a lot of discussion and concern recently about DeepSeek’s models, which claim to rival OpenAI’s, and what that means for the AI ​​landscape. The hype is hard to ignore, so I took the time to analyze the situation, examine what’s really going on under the hood, and compare it to popular models like OpenAI’s o1 and o3.

1. A smarter drive architecture

DeepSeek’s architecture is designed to maximize efficiency without compromising performance. Its sparse attention mechanism allows the model to process only the most relevant data, while ignoring less useful information. Unlike dense attention mechanisms, which analyze all inputs indiscriminately, sparse attention creates an optimized path through the data, ensuring that computational resources are used where they matter most. This significantly reduces computational costs while maintaining accuracy—much like reading only the highlighted passages in a book and still mastering the subject.

In contrast, OpenAI’s o1 and o3 models use dense attention mechanisms, processing all input data, which results in a higher computational load. While this approach provides robust performance, it requires considerable resources. DeepSeek’s approach demonstrates that it is possible to combine efficiency with state-of-the-art performance.

Another key innovation is parameter sharing. DeepSeek reuses its parameters for different tasks via a shared core network, with task-specific adjustments. OpenAI, on the other hand, traditionally favors task-specific adjustment, which increases training time and memory consumption. DeepSeek’s approach reduces redundancy and improves scalability.

2. Training focused on reinforcement learning

DeepSeek-R1 takes a bold approach by prioritizing reinforcement learning (RL) over traditional supervised fine-tuning. This is a major shift because RL allows the model to learn dynamically through trial and error, rather than being limited to massively annotated datasets. Imagine teaching an AI chess by letting it play a large number of games and adapting its strategy along the way, rather than showing it every possible combination up front. This makes RL more flexible and able to adapt to new challenges.

Another key factor is the use of personalized reward signals, which allow the model to optimize its performance in critical areas like problem solving and code generation. OpenAI, on the other hand, relies more on generalized pre-training before fine-tuning per task.

3. A modular design of the model

DeepSeek-R1’s modular architecture allows it to specialize without adding system bulk. Dedicated modules for coding, mathematics, and logical reasoning are integrated into a shared core network, ensuring high performance on various tasks without requiring separate models for each domain.

A particularly innovative element is the meta-controller, which dynamically decides which module to activate depending on the task. This dynamic routing approach differs from OpenAI’s monolithic designs, where a single model handles all tasks without specific optimization. DeepSeek’s modularity ensures efficient use of resources and consistent performance across diverse challenges.

4. An economical computing infrastructure

Rather than relying on traditional GPUs or TPUs, DeepSeek has developed its own accelerators optimized for sparse operations. These accelerators eliminate redundant computation, making them faster and more energy efficient. In contrast, OpenAI’s infrastructure relies on high-end hardware designed for dense operations, but at a much higher cost.

Training was distributed across decentralized clusters, reducing reliance on centralized supercomputers. Asynchronous updates further minimized inefficiencies. This decentralized approach stands in stark contrast to OpenAI’s expensive, centralized infrastructure, illustrating how DeepSeek optimized its process to reduce costs and speed up execution.

5. Strategic use of open-source resources

DeepSeek leveraged open-source datasets, like The Pile, to build a solid foundation without the massive overhead of full pre-training. By using these publicly available datasets, DeepSeek was able to focus its resources on fine-tuning specific tasks, maximizing performance gains where they are most impactful. The diversity and quality of The Pile’s data provided a robust foundation, while DeepSeek’s targeted fine-tuning allowed the model to shine in its advanced use cases, combining efficiency with versatility.

6. Open-source collaboration

DeepSeek’s commitment to open source has been a pillar of its success. By publishing its code on GitHub, the company opened the door to contributions from a global community of developers and researchers. This collaborative approach has accelerated innovation, particularly in areas such as sparse attention optimization and reinforcement learning strategies.

In contrast, OpenAI’s models typically remain closed, limiting outside contributions. DeepSeek’s openness not only democratizes AI development, it also fosters rapid iteration and improvement through knowledge sharing.

7. Practical optimizations for inference

Efficiency in the inference phase was a priority for DeepSeek-R1. Post-training quantization reduced the model size without compromising accuracy, allowing it to run efficiently even on mobile devices. Lightweight runtimes also ensure minimal latency.

OpenAI’s models, while powerful, are often resource-intensive during inference, making them less suitable for low-power devices. DeepSeek’s optimizations improve its accessibility and expand its real-world applications, combining performance and practicality.

Conclusion

DeepSeek’s advances demonstrate the rapid convergence of AI innovation levels. With its focus on efficiency, scalability, and collaboration, DeepSeek challenges costly and resource-intensive Western models. Its ability to achieve cutting-edge results at lower cost marks a major shift in how AI can be developed globally.

The United States maintains an advantage in frameworks, infrastructure, and international partnerships. OpenAI’s models remain benchmarks in terms of performance and versatility. For now…

However, DeepSeek’s approach represents a model for efficient, scalable, and collaborative AI development. By combining sparse attention, reinforcement learning, and modular design, DeepSeek redefines what’s possible with limited resources.

To learn more: discover the vision of Yann LeCun on the World Models and the conference ofArthur Mensch (Mistral).