LLM-Drop

community

AI & ML interests

None defined yet.

LLM-Drop's activity

s1ghhh

updated a collection 3 months ago

LLM-Drop

Model weights of paper "What Matters in Transformers? Not All Attention is Needed" (https://arxiv.org/abs/2406.15786) • 14 items • Updated Oct 23, 2024 • 4

s1ghhh

authored a paper 4 months ago

Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers

Paper • 2410.13184 • Published Oct 17, 2024 • 2

Shwai

authored a paper 4 months ago

Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers

Paper • 2410.13184 • Published Oct 17, 2024 • 2

Shwai

authored 8 papers 7 months ago

Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning

Paper • 2310.11716 • Published Oct 18, 2023 • 5

Merging Experts into One: Improving Computational Efficiency of Mixture of Experts

Paper • 2310.09832 • Published Oct 15, 2023 • 1

Vega-MT: The JD Explore Academy Translation System for WMT22

Paper • 2209.09444 • Published Sep 20, 2022

MerA: Merging Pretrained Adapters For Few-Shot Learning

Paper • 2308.15982 • Published Aug 30, 2023

Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning

Paper • 2402.00530 • Published Feb 1, 2024 • 1

Reformatted Alignment

Paper • 2402.12219 • Published Feb 19, 2024 • 17

Loki: Low-Rank Keys for Efficient Sparse Attention

Paper • 2406.02542 • Published Jun 4, 2024

SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters

Paper • 2210.04284 • Published Oct 9, 2022