Spaces:
Sleeping
Sleeping
File size: 2,110 Bytes
c468e6a c37b091 c468e6a c37b091 c468e6a c37b091 c468e6a c37b091 c468e6a c37b091 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
# You can run models locally or on you own server and use them instead if they are compatible with HuggingFace API
# For local models seletct HF_API as a type because they use HuggingFace API
# Most probalby you don't need a key for your local model
# But if you have some kind of authentication compatible with HuggingFace API you can use it here
HF_API_KEY=None
# The main usecase for the local models in locally running LLMs
# You can serve any model using Text Generation Inference from HuggingFace
# https://github.com/huggingface/text-generation-inference
# It uses Messages API that is compatible with Open AI API and allows you to just plug and play OS models
# Don't forget to add '/v1' to the end of the URL
# Assuming you have Meta-Llama-3-8B-Instruct model running on your local server, your configuration will look like this
LLM_URL=http://192.168.1.1:8080/v1
LLM_TYPE=HF_API
LLM_NAME=Meta-Llama-3-8B-Instruct
# Another polula alternative is ollama https://ollama.com/ it supports the same API and the usage is identical
# OLLAMA_API_KEY=ollama # you don't really need it
# LLM_URL=http://192.168.1.128:11434/v1
# LLM_TYPE=OLLAMA_API
# LLM_NAME=llama3
# Running STT model locally is not straightforward
# But for example you can run one of the whispers models on your laptop
# It requires some simple wrapper over the model to make it compatible with HuggingFace API. Maybe I will share some in the future
# But assuming you manages to run a local whisper-server, your configuration will look like this
STT_URL=http://127.0.0.1:5000/transcribe
STT_TYPE=HF_API
STT_NAME=whisper-base.en
# You can also run TTS models locally without API
# It will use transformers library to run the model
# I would not recommend doing it on the machine without GPU, it will be too slow
# STT_URL=None
# STT_TYPE=HF_LOCAL
# STT_NAME=openai/whisper-base.en
# I don't see much value in running TTS models locally given the quality of online models
# But if you have some kind of TTS model running on your local server you can use it here
TTS_URL=http://127.0.0.1:5001/read
TTS_TYPE=HF_API
TTS_NAME=my-tts-model
|