unsloth

/

DeepSeek-V3-bf16

like 9

Follow

Unsloth AI 1.22k

Dear developers and community users,

I tried to load this unsloth/DeepSeek-V3-bf16 via AutoModelForCausalLM.from_pretrained(" unsloth/DeepSeek-V3-bf16 ", trust_remote_code=True)

But encountered the below error:

Unknown quantization type, got fp8 - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq', 'aqlm', 'quanto', 'eetq', 'hqq', 'compressed-tensors', 'fbgemm_fp8', 'torchao', 'bitnet']

I tried with different transformers versions from 4.33.1, to 4.55, but none worked.

The config file of your model showed

https://huggingface.co/unsloth/DeepSeek-V3-bf16/blob/main/config.json

"torch_dtype": "bfloat16",
"transformers_version": "4.33.1",

Please advise of any fixes? Thanks

ohhh im not sure if hugging face implemented the support for it. :(

Dear developers and community users,

I tried to load this unsloth/DeepSeek-V3-bf16 via AutoModelForCausalLM.from_pretrained(" unsloth/DeepSeek-V3-bf16 ", trust_remote_code=True)

But encountered the below error:

Unknown quantization type, got fp8 - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq', 'aqlm', 'quanto', 'eetq', 'hqq', 'compressed-tensors', 'fbgemm_fp8', 'torchao', 'bitnet']

I tried with different transformers versions from 4.33.1, to 4.55, but none worked.

The config file of your model showed

https://huggingface.co/unsloth/DeepSeek-V3-bf16/blob/main/config.json

"torch_dtype": "bfloat16",
"transformers_version": "4.33.1",

Please advise of any fixes? Thanks

ohhh im not sure if hugging face implemented the support for it. :(

Could you please advise how I can load the model and run it? If HF's AutoModelForCausalLM.from_pretrained() doesn't support it?
thanks.