Update README.md

README.md

@@ -47,4 +47,52 @@ The model was fine-tuned using the following datasets:
 ## Usage
 To use the CodeMind model, you can access it through the Hugging Face model hub or by integrating it into your own applications using the provided API. Provide a coding problem or a question related to programming concepts, and the model will generate relevant explanations, code snippets, or guidance based on its training.
 
-Please refer to the documentation and examples for detailed instructions on how to integrate and use the CodeMind model effectively.
+Please refer to the documentation and examples for detailed instructions on how to integrate and use the CodeMind model effectively.
+
+Below we share some code snippets on how to get quickly started with running the model. After downloading the transformers library via 'pip install -U transformers', use the following snippet code.
+
+#### Running the model on a CPU
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("LimYeri/CodeMind-Gemma-7B-QLoRA-4bit")
+model = AutoModelForCausalLM.from_pretrained("LimYeri/CodeMind-Gemma-7B-QLoRA-4bit")
+
+input_text = "Tell me how to solve the Leetcode Two Sum problem"
+input_ids = tokenizer(input_text, return_tensors="pt")
+
+outputs = model.generate(**input_ids)
+print(tokenizer.decode(outputs[0]))
+```
+
+
+#### Running the model on a single / multi GPU
+
+```python
+# pip install accelerate
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("LimYeri/CodeMind-Gemma-7B-QLoRA-4bit")
+model = AutoModelForCausalLM.from_pretrained("LimYeri/CodeMind-Gemma-7B-QLoRA-4bit")
+
+def get_completion(query: str, model, tokenizer) -> str:
+  device = "cuda:0"
+  prompt_template = """
+  <start_of_turn>user
+  Below is an instruction that describes a task. Write a response that appropriately completes the request.
+  {query}
+  <end_of_turn>\n\n<start_of_turn>model
+
+  """
+  prompt = prompt_template.format(query=query)
+  encodeds = tokenizer(prompt, return_tensors="pt", add_special_tokens=True)
+  model_inputs = encodeds.to(device)
+  generated_ids = model.generate(**model_inputs, max_new_tokens=1000, do_sample=True, pad_token_id=tokenizer.eos_token_id)
+  # decoded = tokenizer.batch_decode(generated_ids)
+  decoded = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
+  return (decoded)
+
+result = get_completion(query="Tell me how to solve the Leetcode Two Sum problem", model=model, tokenizer=tokenizer)
+print(result)
+```