Djuunaa's picture

Djuunaa

djuna

·

AI & ML interests

None yet

Recent Activity

new activity about 7 hours ago

dnhkng/RYS-XLarge:RYS with Qwen2.5

reacted to Elizezen's post with 👀 about 13 hours ago

It turned out that the following simple method seems to be actually effective when you want to increase the appearance probability of only one or a very limited number of tokens. ``` import os one_token = "♡" # Token to increase the appearance probability value = 1000000 token = one_token * value with open("one-token.txt", "w", encoding="utf-8") as f: f.write(token) ``` By training LoRA with unsloth based on the .txt file generated by the code above, you can increase the appearance probability of specific tokens while maintaining the model's performance to great extent. However, it's better to stop the training before train loss becomes 0.0, as it will start spamming the token once it appears even once. In general, you can stop training at a very early stage and it will still work. It is also possible to reduce the appearance probability of specific tokens by creating an over-learned LoRA with the specific tokens you want to reduce, combining it with the model, and then creating a model that extracts only the difference using the chat vector method and subtracting it from an arbitrary model. In this case, it is better to set the ratio of chat vector to about five times. It has very little effect on the overall performance, apart from the specific tokens. ``` new_v = v - (5.0 * chat_vector[i].to(v.device)) ```

liked a model about 13 hours ago

PocketDoc/Dans-PersonalityEngine-V1.1.0-12b

View all activity

Organizations

djuna's activity

New activity in dnhkng/RYS-XLarge about 7 hours ago

RYS with Qwen2.5

#5 opened about 2 months ago by

New activity in djuna/MN-Chinofun-12B-3 1 day ago

Some sample

#3 opened 1 day ago by

New activity in huggingchat/chat-ui 1 day ago

[MODELS] Discussion

#372 opened 10 months ago by

New activity in open-llm-leaderboard/open_llm_leaderboard 2 days ago

14B model detected as 7B

#1049 opened 2 days ago by

New activity in DontPlanToEnd/UGI-Leaderboard 2 days ago

Eval request of Q2.5-Veltha-14B series

#80 opened 2 days ago by

New activity in djuna-test-lab/mergekit-slerp-ynuqykr 3 days ago

broken tokenizer?

#1 opened 3 days ago by

New activity in djuna/Q2.5-Veltha-14B-0.5 3 days ago

Adding Evaluation Results

#2 opened 3 days ago by

Some sample

#1 opened 3 days ago by

New activity in djuna/Q2.5-Veltha-14B 3 days ago

Adding Evaluation Results

#2 opened 3 days ago by

Some sample

#1 opened 4 days ago by

New activity in djuna/G2-Nowing-9B 4 days ago

Some sample

#1 opened 4 days ago by

New activity in relaxml/Llama-3.3-70B-Instruct-QTIP-4Bit 13 days ago

The file isn't merge yet

#2 opened 13 days ago by

New activity in qingy2024/GRMR-2B-Instruct-old 14 days ago

Chat Template problem

#1 opened 14 days ago by

New activity in djuna/MN-Chinofun-12B-3 20 days ago

Low ifeval?

#2 opened 20 days ago by

Adding Evaluation Results

#1 opened 20 days ago by

leaderboard-pr-bot

New activity in AALF/gemma-2-27b-it-SimPO-37K 20 days ago

Difference between this and the other (100 steps) model?

#1 opened 4 months ago by

Adding Evaluation Results

#2 opened 20 days ago by

New activity in AALF/gemma-2-27b-it-SimPO-37K-100steps 20 days ago

Adding Evaluation Results

#2 opened 20 days ago by

New activity in ArliAI/Gemma-2-9B-ArliAI-RPMax-v1.1 21 days ago

New update v1.3?

#1 opened 21 days ago by

New activity in nothingiisreal/MN-12B-Celeste-V1.9 21 days ago

Hoping to start learning

#6 opened 21 days ago by