Unsloth AI

unsloth's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts

danielhanchen 

in unsloth/Mistral-Nemo-Instruct-2407 about 8 hours ago

fix position embeddings

1

#1 opened 1 day ago by

PatentPilotAI

shimmyshimmer 

in unsloth/DeepSeek-V3-bf16 about 12 hours ago

Encountering Unknown quantization type, got fp8 - supported types are: XXXXX

2

#1 opened about 19 hours ago by

ivanmanu

shimmyshimmer 

in unsloth/DeepSeek-V3-GGUF 2 days ago

First review, Q5-K-M require 502Gb RAM, better than Meta 405billions

5

#11 opened 2 days ago by

krustik

shimmyshimmer 

updated 4 models 2 days ago

unsloth/phi-4-GGUF

Text Generation • Updated 2 days ago • 25.4k • 82

unsloth/phi-4-unsloth-bnb-4bit

Text Generation • Updated 2 days ago • 16.6k • 27

unsloth/phi-4

Text Generation • Updated 2 days ago • 9.53k • 57

unsloth/phi-4-bnb-4bit

Text Generation • Updated 2 days ago • 1.04k • 9

shimmyshimmer 

in unsloth/DeepSeek-V3-GGUF 3 days ago

Issue with --n-gpu-layers 5 Parameter: Model Only Running on CPU

4

#10 opened 3 days ago by

vuk123

I’m new to GGUF quants

1

#9 opened 3 days ago by

fsaudm

danielhanchen 

updated a model 4 days ago

unsloth/DeepSeek-V3-GGUF

Updated 4 days ago • 12.5k • 73

danielhanchen 

in unsloth/DeepSeek-V3-GGUF 4 days ago

I loaded DeepSeek-V3-Q5_K_M up on my 10yrs old old Tesla M40 (Dell C4130)

2

#8 opened 5 days ago by

gng2info

shimmyshimmer 

updated a Space 5 days ago

Running

3

🦥

Hi!

shimmyshimmer 

in unsloth/DeepSeek-V3-GGUF 5 days ago

What is the required GPU size to run Is a 4090 possible and does it support ollama

4

#5 opened 7 days ago by

sminbb

danielhanchen 

posted an update 5 days ago

view post

Post

2614

We fixed many bugs in Phi-4 & uploaded fixed GGUF + 4-bit versions! ✨

Our fixed versions are even higher on the Open LLM Leaderboard than Microsoft's!

GGUFs: unsloth/phi-4-GGUF
Dynamic 4-bit: unsloth/phi-4-unsloth-bnb-4bit

You can also now finetune Phi-4 for free on Colab: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Phi_4-Conversational.ipynb

Read our blogpost for more details on bug fixes etc: https://unsloth.ai/blog/phi4

See translation

• 

🔥

10

10

❤️

5

5

😎

4

4

👍

4

4

🤗

3

3

+

Reply

shimmyshimmer 

in unsloth/DeepSeek-V3-GGUF 5 days ago

why use q5 for key cache?

1

#7 opened 5 days ago by

CHNtentes

danielhanchen 

in unsloth/DeepSeek-V3-GGUF 6 days ago

Getting error with Q3-K-M

7

#2 opened 7 days ago by

alain401

Advice on running llama-server with Q2_K_L quant

1

#6 opened 6 days ago by

vmajor

shimmyshimmer 

in unsloth/DeepSeek-V3-GGUF 6 days ago

Are these imatrix GGUF quants?

4

#1 opened 8 days ago by

Kearm

danielhanchen 

in unsloth/DeepSeek-V3-GGUF 6 days ago

llama.cpp cannot load Q6_K model

5

#3 opened 7 days ago by

vmajor

shimmyshimmer 

in unsloth/DeepSeek-V3-GGUF 6 days ago

llama.cpp cannot load Q6_K model

5

#3 opened 7 days ago by

vmajor