Spaces:

YenJung
/

Medical_GPT

Runtime error

YenJung commited on Nov 3, 2023

Commit

061abd2

•

1 Parent(s): f4edf84

Update model.py

Files changed (1) hide show

model.py CHANGED Viewed

@@ -53,7 +53,7 @@ def decide_model(args, device_map):
         tokenizer = _MODEL_CLASSES[model_type].tokenizer.from_pretrained(args.base_model)
         model = _MODEL_CLASSES[model_type].model.from_pretrained(
             args.base_model,
-            load_in_8bit=True,
             torch_dtype=torch.float16,
             device_map=device_map
         )

         tokenizer = _MODEL_CLASSES[model_type].tokenizer.from_pretrained(args.base_model)
         model = _MODEL_CLASSES[model_type].model.from_pretrained(
             args.base_model,
+            load_in_8bit=False,
             torch_dtype=torch.float16,
             device_map=device_map
         )