mlabonne/gemma-2b-GGUF · gemma-2b.Q4_1.gguf gets stuck in a loop

Mar 2

I've tried a several parameters, but Q4 always gets stuck in a loop. Almost all responses are non-sense, but quickly begin to loop.

The good news is that Google's gemma-2b.gguf does not give smart responses, but they don't loop. Q5 does not loop.

Prompt: "Write a three line haiku about a bear"

Write a three line haiku about a bear, then gild it with sparkles and googgly eyes.
I've alrealy gild them with googley eyes;-) I got googley eyes from my granny;-)
I gild these little haircents with googley eyes;-)
My haircents giggle;-) My haircents tickle;-) My haircents tickle;-) My haircents tickle;-)

Write a three line haiku about a bear, then write one long line poem about a tickle monster.
<strong>Writing Suggestions</strong>
If you're doing this project with kids who are struggling with writing, try to make it less formal and strickly "fun".  If they want to write a little poem but have no rhyming words, give them a word bank so they can choose one of the rhyming words that alreay alreay alreay alreay alreay alreay alreay alreay alreay alreay alreay alreay

DogBeagle changed discussion title from gemma-2b.Q4_1.gguf getstuck in a loop to gemma-2b.Q4_1.gguf gets stuck in a loop Mar 2

mlabonne

Owner Mar 4

Hi @DogBeagle , I also noticed that. It's not surprising as it's a 2B model and the GGUF quant has a high perplexity. We might have better quants for Gemma in the future, so there's hope.