AdaptLLM commited on
Commit
d1aafbb
·
verified ·
1 Parent(s): bf17083

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -33,11 +33,11 @@ AdaMLLM is our latest effort to enhance task generalization of (M)LLMs by scalin
33
  - [AdaptLLM](https://huggingface.co/papers/2309.09530)
34
  We employ rule-based methods to extract tasks from domain-specific corpora, reformatting them into reading comprehension tasks for continued pre-training. Our 7B finance model outperforms domain-specific models of much larger scales, such as BloombergGPT-50B.
35
 
36
- - [Instruction Pretraining](https://huggingface.co/papers/2406.14491)
37
- We develop a general-purpose instruction synthesizer which significantly increases task diversity for LM pre-training, outperforming Vanilla Pretraining in both general pretraining from scratch and domain-adaptive continual pretraining.
38
 
39
  - AdaMLLM
40
- We extend supervised task synthesis to multimodality, introducing a unified visual instruction synthesizer to extract task pairs from image-caption data. Our synthetic tasks outperform those generated by manual rules, GPT-4, and GPT-4V in improving domain-specific performance for MLLMs.
41
 
42
  Looking ahead, we envision further broadening the scope of supervised task synthesis, efficiently enhancing the general capabilities of trained models.
43
 
 
33
  - [AdaptLLM](https://huggingface.co/papers/2309.09530)
34
  We employ rule-based methods to extract tasks from domain-specific corpora, reformatting them into reading comprehension tasks for continued pre-training. Our 7B finance model outperforms domain-specific models of much larger scales, such as BloombergGPT-50B.
35
 
36
+ - [Instruction Pre-Training](https://huggingface.co/papers/2406.14491)
37
+ We develop a general-purpose instruction synthesizer which significantly increases task diversity for LM pre-training, outperforming vanilla pre-training in both general pre-training from scratch and domain-adaptive continual pre-training.
38
 
39
  - AdaMLLM
40
+ We extend supervised task synthesis to multimodality, introducing a unified visual instruction synthesizer to extract instruction-response pairs from image-caption data. Our synthetic tasks outperform those generated by manual rules, GPT-4, and GPT-4V in improving domain-specific performance for MLLMs.
41
 
42
  Looking ahead, we envision further broadening the scope of supervised task synthesis, efficiently enhancing the general capabilities of trained models.
43