Update README.md

README.md

@@ -1,3 +1,70 @@
 ---
 license: mit
+language:
+- en
+thumbnail: "https://cdn-uploads.huggingface.co/production/uploads/6589d7e6586088fd2784a12c/2g6TFg4nLSigVy7n1wLYV.png"
 ---
+# Done on 64GB of system RAM, and a Ryzen 5 5600...no GPU needed
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/6589d7e6586088fd2784a12c/2g6TFg4nLSigVy7n1wLYV.png)
+
+### Passthrough was used to create this model.
+
+
+[Kronos](https://www.theoi.com/Titan/TitanKronos.html) was a titan, and this model is named after him for the sheer size of the model.
+
+By concatenating layers from different LLMs, the method used for this model, passthrough. can produce models with an exotic number of parameters (e.g., 9B with two 7B parameter models). These models are often referred to as "frankenmerges" or "Frankenstein models" by the community.
+
+
+Many thanks to [Abacaj](https://huggingface.co/abacaj) for providing the [fine tuned weights](https://huggingface.co/abacaj/phi-2-super) that were used in the creation of this base model. You can find the full script for how the model was merged [here](https://huggingface.co/Replete-AI/Phi-Elothir/blob/main/mergekit_config.yml)...thanks to [KatyTheCutie](https://huggingface.co/KatyTheCutie) for inspring me to test out this script.
+
+## This idea was brought to me by [The Face of Goonery](https://huggingface.co/The-Face-Of-Goonery), also known as Caleb Morgan.
+# How to run inference:
+
+```python
+import transformers
+import torch
+
+if __name__ == "__main__":
+ model_name = "Replete-AI/Kronos-670B"
+ tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)
+ 
+ model = (
+ transformers.AutoModelForCausalLM.from_pretrained(
+ model_name,
+ )
+ .to("cuda:0")
+ .eval()
+ )
+ 
+ messages = [
+ {"role": "user", "content": "Hello, who are you?"}
+ ]
+ inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
+ input_ids_cutoff = inputs.size(dim=1)
+ 
+ with torch.no_grad():
+ generated_ids = model.generate(
+ input_ids=inputs,
+ use_cache=True,
+ max_new_tokens=512,
+ temperature=0.2,
+ top_p=0.95,
+ do_sample=True,
+ eos_token_id=tokenizer.eos_token_id,
+ pad_token_id=tokenizer.pad_token_id,
+ )
+ 
+ completion = tokenizer.decode(
+ generated_ids[0][input_ids_cutoff:],
+ skip_special_tokens=True,
+ )
+ 
+ print(completion)
+```
+
+# Chat template
+
+The model uses the same chat template as found in Mistral instruct models:
+
+# [Join the Replete AI Discord here!](https://discord.gg/tG5aY4EX4T)