llm-stacking (llm-stacking)

Kuroxiro

authored a paper 7 months ago

Unlocking Continual Learning Abilities in Language Models

Paper • 2406.17245 • Published Jun 25, 2024 • 29

wydu

authored 2 papers 7 months ago

Unlocking Continual Learning Abilities in Language Models

Paper • 2406.17245 • Published Jun 25, 2024 • 29

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

Paper • 2405.15319 • Published May 24, 2024 • 25

wydu

updated 12 models 7 months ago

wydu

updated a Space 7 months ago

Running

🌖

README

wydu

updated 4 models 7 months ago

llm-stacking/Ablation_PartialStack_123x7-456

Updated Jun 5, 2024 • 1

llm-stacking/Ablation_PartialStack_1-234x7-56

Updated Jun 5, 2024 • 1

llm-stacking/Ablation_PartialStack_12-34x10-56

Updated Jun 5, 2024 • 1

llm-stacking/Ablation_PartialStack_1234-56x10

Updated Jun 5, 2024 • 1

llm-stacking

AI & ML interests

llm-stacking's activity

Unlocking Continual Learning Abilities in Language Models

Unlocking Continual Learning Abilities in Language Models

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training

llm-stacking/StackingLaw_Factor_1.1B12L

llm-stacking/StackingLaw_Factor_1.1B3L

llm-stacking/StackingLaw_Factor_1.1B1L

llm-stacking/StackingLaw_Timing_1.1B50BT

llm-stacking/StackingLaw_Timing_1.1B20BT

llm-stacking/StackingLaw_Timing_1.1B5BT

llm-stacking/StackingLaw_Timing_1.1B1BT

llm-stacking/StackingLaw_Timing_410M50BT

llm-stacking/StackingLaw_Timing_410M20BT

llm-stacking/StackingLaw_Timing_410M10BT

llm-stacking/StackingLaw_Timing_410M5BT

llm-stacking/StackingLaw_Timing_410M1BT

README

llm-stacking/Ablation_PartialStack_123x7-456

llm-stacking/Ablation_PartialStack_1-234x7-56

llm-stacking/Ablation_PartialStack_12-34x10-56

llm-stacking/Ablation_PartialStack_1234-56x10

AI & ML interests

Team members 2

llm-stacking's activity

README