Q-bert
/

MambaHermes-3B

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Q-bert commited on Dec 28, 2023

Commit

0173525

·

1 Parent(s): bdd4211

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -32,7 +32,7 @@ tokenizer.pad_token = tokenizer.eos_token
 tokenizer.chat_template = AutoTokenizer.from_pretrained(CHAT_TEMPLATE_ID).chat_template
 model = AutoModelForCausalLM.from_pretrained(
-        model_name, device=device, trust_remote_code=True, dtype=torch.float16)
 messages = []
 prompt = "Tell me 5 sites to visit in Spain"

 tokenizer.chat_template = AutoTokenizer.from_pretrained(CHAT_TEMPLATE_ID).chat_template
 model = AutoModelForCausalLM.from_pretrained(
+        model_name, device_map=device, trust_remote_code=True)
 messages = []
 prompt = "Tell me 5 sites to visit in Spain"