microsoft
/

bloom-deepspeed-inference-fp16

Feature Extraction

Inference Endpoints

Model card Files Files and versions Community

jeffra commited on Aug 30, 2022

Commit

51dd2ae

·

1 Parent(s): 5923650

add dtype to config

Files changed (1) hide show

ds_inference_config.json +2 -1

ds_inference_config.json CHANGED Viewed

@@ -73,5 +73,6 @@
   },
   "version": 1,
   "parallelization": "tp",
-  "mp_size": 8
 }

   },
   "version": 1,
   "parallelization": "tp",
+  "mp_size": 8,
+  "dtype": "float16"
 }