It’s not only a proposal but an announcement that they are trying to do it.
ericjmorey@programming.dev
on 28 Jan 14:09
nextcollapse
From the article:
DeepSeek-R1 release leaves open several questions about:
Data collection: How were the reasoning-specific datasets curated?
Model training: No training code was released by DeepSeek, so it is unknown which hyperparameters work best and how they differ across different model families and scales.
Scaling laws: What are the compute and data trade-offs in training reasoning models?
These questions prompted us to launch the Open-R1 project, an initiative to systematically reconstruct DeepSeek-R1’s data and training pipeline, validate its claims, and push the boundaries of open reasoning models. By building Open-R1, we aim to provide transparency on how reinforcement learning can enhance reasoning, share reproducible insights with the open-source community, and create a foundation for future models to leverage these techniques.
In this blog post we take a look at key ingredients behind DeepSeek-R1, which parts we plan to replicate, and how to contribute to the Open-R1 project
threaded - newest
This is just a proposal to make an open reproduction, nobody actually did it yet.
It’s not only a proposal but an announcement that they are trying to do it.
From the article:
DeepSeek-R1 release leaves open several questions about:
These questions prompted us to launch the Open-R1 project, an initiative to systematically reconstruct DeepSeek-R1’s data and training pipeline, validate its claims, and push the boundaries of open reasoning models. By building Open-R1, we aim to provide transparency on how reinforcement learning can enhance reasoning, share reproducible insights with the open-source community, and create a foundation for future models to leverage these techniques.
In this blog post we take a look at key ingredients behind DeepSeek-R1, which parts we plan to replicate, and how to contribute to the Open-R1 project
All I want is a 3gb model for the raspberry pi. 7b is too big and 1.5b is too stupid.
3B is probably also pretty dumb
honestly both 7b and 8b are pretty dumb as well.
True
we could add so much deterministic code at 1.5GB that would start religions…
that’s why gen ai models are not “open source”, ever. If they were, this group would’t have to “try”, they could just run the build script.
Of course, the training data and software is not available. The weights are just a binary blob. It’s not the source, but merely the “compiled binary”