---
license: llama3.1
datasets:
- survivi/Llama-3-SynE-Dataset
- hfl/stem_zh_instruction
- llamafactory/alpaca_zh
- llamafactory/alpaca_gpt4_zh
- hfl/ruozhiba_gpt4
- codingsteven/Llama-3-8B-chat
language:
- zh
metrics:
- accuracy
base_model:
- meta-llama/Llama-3.1-8B
pipeline_tag: text-generation
library_name: transformers
model-index:
- name: Control-LLM-Llama3.1-8B-SynE-Concat16-Dlerp
  results:
  - task:
      type: pretraining-evaluation
    dataset:
      type: mixed
      name: Pretraining Evaluation Dataset
    metrics:
    - name: exact_match,strict-match (meta_pretrain)
      type: exact_match
      value: 0.48514264142803215
      stderr: 0.003513307445696379
      verified: false
    - name: exact_match,strict-match (meta_bbh_3shot_cot_pretrain)
      type: exact_match
      value: 0.6817693134695131
      stderr: 0.0057729694388110805
      verified: false
    - name: acc,none (meta_mmlu_5shot_pretrain)
      type: accuracy
      value: 0.65596068936049
      stderr: 0.0040090726054856874
      verified: false
    - name: exact_match,strict-match (meta_mmlu_pro_5shot_pretrain)
      type: exact_match
      value: 0.3787400265957447
      stderr: 0.004422383756050139
      verified: false
  - task:
      type: chinese-evaluation
    dataset:
      type: mixed
      name: Chinese Evaluation Dataset
    metrics:
    - name: exact_match,strict-match (zh_pretrain_multishot)
      type: exact_match
      value: 0.44848391089108913
      stderr: 0.004255614019851072
      verified: false
    - name: acc,none (ceval-valid)
      type: accuracy
      value: 0.5698365527488856
      stderr: 0.012893833892221353
      verified: false
    - name: exact_match,strict-match (ceval-valid-pretrain-cot_zh)
      type: exact_match
      value: 0.4472511144130758
      stderr: 0.013203606600472227
      verified: false
    - name: acc,none (cmmlu)
      type: accuracy
      value: 0.5602659298912105
      stderr: 0.0044928840587441605
      verified: false
    - name: exact_match,strict-match (cmmlu_pretrain_cot_zh)
      type: exact_match
      value: 0.4486271801070627
      stderr: 0.00449553418468653
      verified: false
---

# Control-LLM-Llama3.1-8B-SynE-Concat16-Dlerp
This is a fine-tuned model of Llama-3.1-8B for muliligual-Chinese tasks on SynE dataset by Control LLM-Concat16-Dlerp, as described in [Control LLM: Controlled Evolution for Intelligence Retention in LLM](https://huggingface.co/papers/2501.10979).

## Linked Paper
This model is associated with the paper: [Control-LLM](https://arxiv.org/abs/2410.14745).

## Linked Open Source code - training, eval and benchmark
This model is associated with the github: [Control-LLM](https://github.com/linkedin/ControlLLM).

## Evaluation Results
Here is an overview of the evaluation results and findings:

### Benchmark Results Table

The table below summarizes evaluation results across Chinese tasks and original capabilities.

| **Model**          | **CEval** | **CEvalC** | **CMMLU** | **CMMLUC** | **C-Avg** | **BBH** | **MLU** | **MLUP** | **O-Avg** | **Overall** |
|--------------------|-----------|------------|-----------|------------|-----------|---------|---------|----------|-----------|-------------|
| Llama3.1-8B        | 48.3      | 12.8       | 51.1      | 14.1       | 13.9      | 65.2    | 65.4    | 35.5     | 45.9      | 29.9        |
| Llama-3-SynE       | 57.7      | 22.3       | 57.1      | 22.8       | 22.8      | 61.9    | 64.0    | 32.6     | 42.9      | 32.9        |
| Full Param Tune   | 59.0      | 40.2       | **60.2**  | 44.3       | 43.8      | 64.8    | 64.9    | 35.0     | 45.4      | 44.6        |
| Stack Expansion    | 56.0      | 32.7       | 55.2      | 33.4       | 33.3      | 62.3    | 65.6    | 35.3     | 44.8      | 39.1        |
| Concat-Lerp       | 57.1      | 34.8       | 57.0      | 37.4       | 37.1      | 64.4    | 64.6    | 35.8     | 45.9      | 41.5        |
| Hybrid Expansion  | **58.9** | 44.7       | 57.9      | 44.3       | 44.4      | 65.1    | **65.7**| 36.9     | 46.8      | 45.6        |
| **Control LLM***   | 57.0      | **44.7**   | 56.0      | **44.9**   | **44.8**  | **68.2**| 65.6    | **37.9** | **48.5**  | **46.7**    |

---

### Explanation:
- **CEval**: Chinese Evaluation
- **CEvalC**: Chinese Evaluation (CoT - Chain of Thought)
- **CMMLU**: Chinese MMLU
- **CMMLUC**: Chinese MMLU (CoT)
- **C-Avg**: Chinese - Size Weighted Average across CEval, CEvalC, CMMLU, and CMMLUC
- **BBH**: BigBench Hard
- **MLU**: MMLU (Massive Multitask Language Understanding)
- **MLUP**: MMLU Pro
- **O-Avg**: Original Capability - Size Weighted Average across BBH, MLU, and MLUP
- **Overall**: Combined average across all tasks