Local use

#7
by Scoppy - opened

Hi! How can I use it in local ? I use LM Studio as a provider with Gemma3 and Deepseek-Qwen.

Hi! How can I use it in local ? I use LM Studio as a provider with Gemma3 and Deepseek-Qwen.

The code is right there under Files, clone it and toss it at your favorite AI and see how it goes. That should be a good place to start for any question like that.

I had it to run but I don't know how to set my OpenAI like api as provider. I never touched react...

Hi @Scoppy
You can find the code related to the generation there: (javascript) https://huggingface.co/spaces/enzostvs/deepsite/blob/4231ffc7c4c5bdcfed7ca55df0813cd0a1e65423/server.js#L219

You can adapt it to fit your idea ๐Ÿ’ก

I never touched JS so I don't know how to do it. So if someone could put me in the way. I had first, before answer, tried to modify ask-ai but I got an error about '
'messages' field is required' in LM Studio console
Here are my endpoint and my provider is the local host : http://127.0.0.1:1234
image.png

Copy/Paste this into a CMD terminal:

docker run -it -p 5173:5173 -e GRADIO_SERVER_PORT=5173 -e GRADIO_ALLOW_FLAGGING="never" -e SPACE_ID="new" -e HF_TOKEN="This is your Huggingface Key" -e APP_PORT="5173" -e REDIRECT_URI="https://enzostvs-deepsite.hf.space/auth/login" --platform=linux/amd64 --gpus all registry.hf.space/enzostvs-deepsite:latest

Your Welcome!

Copy/Paste this into a CMD terminal:

docker run -it -p 5173:5173 -e GRADIO_SERVER_PORT=5173 -e GRADIO_ALLOW_FLAGGING="never" -e SPACE_ID="new" -e HF_TOKEN="This is your Huggingface Key" -e APP_PORT="5173" -e REDIRECT_URI="https://enzostvs-deepsite.hf.space/auth/login" --platform=linux/amd64 --gpus all registry.hf.space/enzostvs-deepsite:latest

Your Welcome!

Thanks but it will only work with HF, no LM Studio, will it ?

Make these changes to get it to run locally with LM Studio and the model you have loaded into its dev console etc:

DockerFile (You may not have to change this):
https://gist.github.com/TheyGiveMeMoney/076d3cc4250b367009ec0d67c9ffbea4

PackageJSON (make sure to run npm install axios):
https://gist.github.com/TheyGiveMeMoney/8ad27b044cb16ff3bd4c9c25916f187e
(verify your package.json looks like this, or similar - just looking for active axios install)

server.js:
https://gist.github.com/TheyGiveMeMoney/dfa7e476b19788680b0afb2ea7b3ce6f
(This uses default http://127.0.0.1:1234 from LM Studio)

providers.js:
https://gist.github.com/TheyGiveMeMoney/58633b1b3aa12556fefd03f6d2b36e1f
(Here you are just adding LM Studio to the list of providers for the Application)

SAVE>COMPILE>DOCKER RUN IT

Confirm if it works for you!

GoodLuck HappyCoding

EDIT: If having trouble where app says -> connect ECONNREFUSED 127.0.0.1:1234

USE:

docker run -p 3000:3000 --add-host=host.docker.internal:host-gateway deepsite

(to run the app, change your port to whatever you have configured if different)

Make these changes to get it to run locally with LM Studio and the model you have loaded into its dev console etc:

DockerFile (You may not have to change this):
https://gist.github.com/TheyGiveMeMoney/076d3cc4250b367009ec0d67c9ffbea4

PackageJSON (make sure to run npm install axios):
https://gist.github.com/TheyGiveMeMoney/8ad27b044cb16ff3bd4c9c25916f187e
(verify your package.json looks like this, or similar - just looking for active axios install)

server.js:
https://gist.github.com/TheyGiveMeMoney/dfa7e476b19788680b0afb2ea7b3ce6f
(This uses default http://127.0.0.1:1234 from LM Studio)

providers.js:
https://gist.github.com/TheyGiveMeMoney/58633b1b3aa12556fefd03f6d2b36e1f
(Here you are just adding LM Studio to the list of providers for the Application)

SAVE>COMPILE>DOCKER RUN IT

Confirm if it works for you!

GoodLuck HappyCoding

EDIT: If having trouble where app says -> connect ECONNREFUSED 127.0.0.1:1234

USE:

docker run -p 3000:3000 --add-host=host.docker.internal:host-gateway deepsite

(to run the app, change your port to whatever you have configured if different)

Hi, thanks a lot, I did everything and it's working perfectly... Thanks a lot, really.


docker run -p 3000:3000 --add-host=host.docker.internal:host-gateway deepsite

(to run the app, change your port to whatever you have configured if different)

Hi, thanks a lot, I did everything and it's working perfectly... Thanks a lot, really.

Cool Beans, enjoy ๐Ÿ‘

enzostvs changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment