serpdotai
/

sparsetral-16x7B-v2

Text Generation

Inference Endpoints

Model card Files Files and versions Community

francislabounty commited on Feb 5

Commit

03c7d6f

•

1 Parent(s): c6d1699

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -53,4 +53,5 @@ prompt = construct_prompt(messages)
 inputs = tokenizer(prompt, return_tensors="pt")
 inputs = inputs.to(model.device)
 pred = model.generate(**inputs, max_length=4096, do_sample=True, top_k=50, top_p=0.99, temperature=0.9, num_return_sequences=1)
-print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))

 inputs = tokenizer(prompt, return_tensors="pt")
 inputs = inputs.to(model.device)
 pred = model.generate(**inputs, max_length=4096, do_sample=True, top_k=50, top_p=0.99, temperature=0.9, num_return_sequences=1)
+print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
+```