Update README.md
Browse files
README.md
CHANGED
@@ -84,7 +84,9 @@ model = AutoModelForCausalLM.from_pretrained("uiuc-convai/CALM-8B")
|
|
84 |
```
|
85 |
|
86 |
### 🛠 Example Oumi Inference
|
87 |
-
|
|
|
|
|
88 |
|
89 |
### 🛠 Example Oumi Fine-Tuning
|
90 |
```bash
|
|
|
84 |
```
|
85 |
|
86 |
### 🛠 Example Oumi Inference
|
87 |
+
Oumi multi-node inference support is under development.
|
88 |
+
CALM-405B likely requires multi-node inference as most single nodes support up to 640GB of GPU VRAM.
|
89 |
+
To run multi-node inference, we recommend [vLLM](https://docs.vllm.ai/en/latest/serving/distributed_serving.html)
|
90 |
|
91 |
### 🛠 Example Oumi Fine-Tuning
|
92 |
```bash
|