Spaces:

PingAndPasquale
/

med-rag

Sleeping

pminervini commited on Mar 2, 2024

Commit

ee28881

1 Parent(s): 238b842

update

Files changed (1) hide show

app.py CHANGED Viewed

@@ -122,7 +122,6 @@ def rag_pipeline(prompt, index="pubmed", num_docs=3, model_name="HuggingFaceH4/z
         response = openai_res.choices[0].message.content.strip()
     else:
         quantization_config = BitsAndBytesConfig(load_in_4bit=True)
         model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto", low_cpu_mem_usage=True, quantization_config=quantization_config)
         tokenizer = AutoTokenizer.from_pretrained(model_name)

         response = openai_res.choices[0].message.content.strip()
     else:
         quantization_config = BitsAndBytesConfig(load_in_4bit=True)
         model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto", low_cpu_mem_usage=True, quantization_config=quantization_config)
         tokenizer = AutoTokenizer.from_pretrained(model_name)