Hugging Face's logo

Spaces:
Daemontatox
/
Smart_LLM
Running on Zero

App Files Community
1
Daemontatox commited on
Commit
a6ea901
·
verified ·
1 Parent(s): 2fd228d

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +1 -1
app.py CHANGED
@@ -80,7 +80,7 @@ def initialize_model():
80
  model = AutoModelForCausalLM.from_pretrained(
81
  MODEL_ID,
82
  torch_dtype=torch.float16,
83
- device_map="auto",
84
  attn_implementation="flash_attention_2",
85
  # quantization_config=quantization_config
86
 
 
80
  model = AutoModelForCausalLM.from_pretrained(
81
  MODEL_ID,
82
  torch_dtype=torch.float16,
83
+ device_map="cuda",
84
  attn_implementation="flash_attention_2",
85
  # quantization_config=quantization_config
86