ChatGPT-o1 Destroyed by Chinese Quant Shop
Introduction
I’m going to change my writing style to output shorter articles faster because the rate of improvement in AI is unbelievable. As I was finishing up one of my usual deep-dives (publishing next week), a chinese quant shop just destroyed ChatGPT-o1
Here’s what happened:
DeepSeek R1
The News
DeepSeek-R1 is an LLM released by a group of quants working on an OpenAI competitor as a hobby project:
The reason this model has been making waves is because they beat OpenAI by a substantial margin in terms of efficiency. It is not only on par with OpenAI-o1, but it is also fully open-sourced, has a technical report, and is MIT licensed which means it can be used freely without copyright infringement:
Now why would you use this instead of ChatGPT-o1? For three reasons:
It’s over 30x cheaper for the same performance, without any rate-limiting:
It’s fully open-source, which means you can run it on your own device, without internet access, for free:
You can read how DeepSeek does everything, from the research to the actual model’s thinking. When you use ChatGPT-o1, notice how it says “Thinking…” but it doesn’t actually tell you how it’s thinking? DeepSeek reveals all of this to you, so you can better verify whether it’s correct, and learn how it reasons. It tells you how it argues with itself, tests ideas, and refines its approach just like a human:
Bonus: It has access to more high-quality data on Chinese literature, history, and philosophy:
As of 2024-01-26 it just hit #1 on the App Store right above ChatGPT:
My Thoughts
My takeaway from the research paper is that AI models improve their reasoning ability in general when you teach them maths and programming. These two subjects are the easiest to teach AI because we can easily check the end result. A simple calculator can evaluate 2 + 2 = 4
I wonder what this says about humans? Do they become better reasoners when you teach them mathematics?
I think yes, for two reasons:
First, because decisions require tradeoffs. Optimal decisions calculate the payoff mathematically between two choices
“Should I quit my job to pursue my business?” This is an easy decision for someone who can run the numbers: what are the odds that my job becomes easier/harder as my business improves/worsens?
Second, because you are forced to use rules of logic. You need to understand:
a. Assumptions: Do I have everything I need to solve this problem? Or do I need to figure out the measurements for this triangle first?
b. Rules: When I add, multiply, subtract, divide, am I following the exact same steps every single time to achieve the same result?
c. Soundness: Can someone follow my reasoning to get to my conclusion, and repeat it every single time?
d. Clarity: Do I understand the problem fully, and know what is unknown?
e. Breakdowns: Can I use smaller steps to make this problem easier to fix?
f. Strategy: What are the different ways that I can solve this problem?
And each of these skills improve your general problem-solving ability.