Hugging Face's logo

deepseek-ai
/
DeepSeek-V3-Base

Safetensors
deepseek_v3
custom_code
fp8
Model card Files Community
59

`aux_loss_alpha` should be 1e-4 instead of 1e-3?

#60
by cuichenx - opened
base: refs/heads/main
←
from: refs/pr/60
Discussion Files changed
+1
-1
cuichenx

According to DeepSeekV3 technical report section 4.2

For the balance loss, we set 𝛼 to 0.0001

`aux_loss_alpha` should be 1e-4 instead of 1e-3?41955df7
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment