Update README.md
Browse files
README.md
CHANGED
@@ -33,11 +33,11 @@ AdaMLLM is our latest effort to enhance task generalization of (M)LLMs by scalin
|
|
33 |
- [AdaptLLM](https://huggingface.co/papers/2309.09530)
|
34 |
We employ rule-based methods to extract tasks from domain-specific corpora, reformatting them into reading comprehension tasks for continued pre-training. Our 7B finance model outperforms domain-specific models of much larger scales, such as BloombergGPT-50B.
|
35 |
|
36 |
-
- [Instruction
|
37 |
-
We develop a general-purpose instruction synthesizer which significantly increases task diversity for LM pre-training, outperforming
|
38 |
|
39 |
- AdaMLLM
|
40 |
-
We extend supervised task synthesis to multimodality, introducing a unified visual instruction synthesizer to extract
|
41 |
|
42 |
Looking ahead, we envision further broadening the scope of supervised task synthesis, efficiently enhancing the general capabilities of trained models.
|
43 |
|
|
|
33 |
- [AdaptLLM](https://huggingface.co/papers/2309.09530)
|
34 |
We employ rule-based methods to extract tasks from domain-specific corpora, reformatting them into reading comprehension tasks for continued pre-training. Our 7B finance model outperforms domain-specific models of much larger scales, such as BloombergGPT-50B.
|
35 |
|
36 |
+
- [Instruction Pre-Training](https://huggingface.co/papers/2406.14491)
|
37 |
+
We develop a general-purpose instruction synthesizer which significantly increases task diversity for LM pre-training, outperforming vanilla pre-training in both general pre-training from scratch and domain-adaptive continual pre-training.
|
38 |
|
39 |
- AdaMLLM
|
40 |
+
We extend supervised task synthesis to multimodality, introducing a unified visual instruction synthesizer to extract instruction-response pairs from image-caption data. Our synthetic tasks outperform those generated by manual rules, GPT-4, and GPT-4V in improving domain-specific performance for MLLMs.
|
41 |
|
42 |
Looking ahead, we envision further broadening the scope of supervised task synthesis, efficiently enhancing the general capabilities of trained models.
|
43 |
|