nicolo's picture

nicolo

nicolollo

·

AI & ML interests

None yet

Recent Activity

liked a Space about 1 month ago

fancyfeast/joy-caption-alpha-two

liked a model about 2 months ago

nomic-ai/nomic-embed-text-v2-moe

reacted to grimjim's post with ❤️ 2 months ago

This recent paper points to an explanation for the unreasonable effectiveness of Frankenmerges: https://huggingface.co/papers/2502.05171 Specifically, the duplication of layers in Frankenmerges serves a purpose similar to what occurs in their recurrent-depth architecture. Successful frankenmerges that operate without additional fine-tuning are able to recover or "heal" from any damage due to abrupt transitions between layer blocks. Operational replicated layer blocks can provide functional benefits grounded in latent reasoning. Frankenmerges can also result in hybrid reasoning, by splicing together the latent reasoning of different models. Back in April 2024, I was able to duplicate a few layers in the Llama 3 8B model, turning it into a 9B model, without harming benchmarks significantly, despite any transition damage. https://huggingface.co/grimjim/llama-3-experiment-v1-9B My informal experimentation suggested that latent reasoning circuits could occupy continguous stacks of 2-4 layers, though the result was highly sensitive to the choice of transition location between layers.

View all activity

Organizations

nicolollo's activity

New activity in huihui-ai/Qwen2-VL-7B-Instruct-abliterated 6 months ago

Request gguf

#1 opened 7 months ago by

New activity in Qwen/Qwen2.5-3B 7 months ago

License

#1 opened 7 months ago by

New activity in Qwen/Qwen2-VL-7B-Instruct 8 months ago

LoRA Finetuning Tool for Qwen2-VL-7B in Web UI (DPO updated)

#2 opened 8 months ago by

New activity in qnguyen3/nanoLLaVA-1.5 8 months ago

This model is amazing!

#1 opened 9 months ago by

New activity in NikshepShetty/Florence-2-DOCCI-FT 8 months ago

Target module question

#1 opened 8 months ago by

New activity in bczhou/TinyLLaVA-1.5B 8 months ago

Finetuning GPU Requirement

#1 opened 12 months ago by

New activity in ljnlonoljpiljm/florence-2-large-interleaved-docci-keywords 9 months ago

any custom data ?

#1 opened 9 months ago by

New activity in cognitivecomputations/dolphin-vision-7b 9 months ago

The checkpoint you are trying to load has model type `bunny-qwen` but Transformers does not recognize this architecture.

#3 opened 9 months ago by

Transformers doesn't support it yet?

#2 opened 9 months ago by

New activity in ljnlonoljpiljm/paligemma_docci_caption_enriched_v1 10 months ago

WOW

#1 opened 10 months ago by

New activity in gokaygokay/Florence-2 10 months ago

Thanks you

#1 opened 10 months ago by

New activity in HuggingFaceM4/idefics2-8b 12 months ago

What is the minimum number of examples to fine-tune IDEFICS-2 to get the expected output?

#14 opened about 1 year ago by

New activity in meta-llama/Meta-Llama-3-8B 12 months ago

Mosaic

#49 opened 12 months ago by

New activity in vikhyatk/moondream2 about 1 year ago

Qlora fine-tune

#7 opened about 1 year ago by

New activity in huggingchat/chat-ui about 1 year ago

[MODELS] Discussion

#372 opened about 1 year ago by

[MODELS] Discussion

#372 opened about 1 year ago by