c

README.md

@@ -4,7 +4,16 @@ base_model: mistralai/Mistral-7B-v0.3
 extra_gated_description: If you want to learn more about how we process your personal data, please read our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
 ---
 
-# Model Card for Mistral-7B-Instruct-v0.3
+# Model Card for Mistral-7B-Instruct-v0.3 for inf2.xlarge
 
 The Mistral-7B-Instruct-v0.3 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.3.
 neuron compiled
+these are the shapes currently cached. 
+don't ask me why 8196.. it was a typo.
+   ``` 
+   python -m vllm.entrypoints.openai.api_server --model ./  --max-model-len 8196 --device neuron --tensor-parallel-size 2 --max-num-seqs 2
+   python -m vllm.entrypoints.openai.api_server --model ./  --max-model-len 8196 --device neuron --tensor-parallel-size 2 --max-num-seqs 4
+   python -m vllm.entrypoints.openai.api_server --model ./  --max-model-len 10240 --device neuron --tensor-parallel-size 2 --max-num-seqs 4
+```
+
+