Implement other approach that optillm supported?

#5
by melekuk - opened

Hi, this spaces is cool. But it's don't have some of new approach. Would be nice if it implemented.

Actually all the supported approaches are available in the GUI. Other approaches from https://github.com/codelion/optillm?tab=readme-ov-file#implemented-techniques either require sampling multiple responses (like mcts, moa, cepo, pvg) or require decoding during generation (cot_decoding, entropy_decoding, thinkdeeper) and thus cannot be implemented with a proxy.

We do have a simple chat based GUI in the project that you can use with any of the approaches to test locally.

python optillm.py --launch-gui TRUE --model mcp-google/gemini-2.0-flash-001 --log debug --base-url https://openrouter.ai/api/v1

E.g. above will launch the gradio GUI connected to optillm using the model gemini from open router with the mcp approach (model context protocol plugin).

I see. Thanks ๐Ÿ™

melekuk changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment