Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
ngxson 
posted an update 1 day ago
Post
779
Check out my collection of pre-made GGUF LoRA adapters!

This allow you to use both normal + abliterated version of popular models like llama, qwen, etc, without having to double to amount of VRAM usage.

ngxson/gguf_lora_collection
ngxson

Tagging @bartowski @MaziyarPanahi and @mradermacher , you may want to give this a try!

John6666
•
edited about 22 hours ago

With my llama-cpp-python (0.3.4), the following PR maybe have not been merged yet, so an error occurs when applying LoRA. I tried it with Qwen 2.5 14B Instruct. Well, it will be updated eventually.🙄
https://github.com/ggerganov/llama.cpp/issues/9114

MaziyarPanahi

This is super cool!!! Would you mind sharing the process of these GGUF LoRA adapters? Did you convert the LoRA into GGUF or made LoRA from the GGUF itself?

In this post