Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
Smoke666 
posted an update Jun 21
Post
663
@Be-Bo
Dear Mr. Bahaa Shamoon Atia,
My name is Krischan Schoeninger, and I am very impressed with your Llama 3-70B Chatbot that you have made available on Hugging Face. I have been trying to use both your chatbot and the model from Hugging Face via API for a project, and I have found that your model produces significantly better results.
Could you please let me know what changes or optimizations you have made to your model that make it so powerful? Additionally, I am very interested in learning how I can host such a model myself. Could you assist me with this?
I would greatly appreciate your feedback.
Best regards,
Krischan Schoeninger
YaTharThShaRma999

I’m obviously not Mr. Bahaa Shamoon Atia but could you show some example where it performs worse. A lot of issues is usually chat template which hf should do correctly I believe.

·
Smoke666

Dear Yatharth Sharma,

I apologize if I used the wrong name. It seemed to me that I was using the correct one.

I have used the LLaMA-3-70B-Instruct model from Meta on Hugging Face via API and the variant on the Be-Bo Space:

Meta API: https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-70B-Instruct
Be-Bo Space: https://huggingface.co/spaces/Be-Bo/llama-3-chatbot_70b
The Meta variant often produced nonsensical outputs and did not follow the instructions correctly. The Be-Bo variant responded correctly except for very minor errors. Could you help me run the Be-Bo variant myself?

Best regards,

Krischan Schoeninger

Artples
•
edited Jun 21

From the code, it looks like he uses Llama 70b hosted with Groq with following parameters: max_tokens=1024, temperature=1.3 and as a system prompt: You are a useful assistant. You reply with efficient answers.

kargaranamir
•
edited Jun 21

Please use discussions for these kind of things: https://huggingface.co/spaces/Be-Bo/llama-3-chatbot_70b/discussions not a post.

In this post