Hugging Face's logo

OpenGVLab
/
InternVL2-Llama3-76B

Image-Text-to-Text
Transformers
Safetensors
multilingual
internvl_chat
feature-extraction
internvl
vision
ocr
multi-image
video
custom_code
conversational
Model card Files Community
11
czczup commited on
Commit
d640123
•
1 Parent(s): e779c52

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +46 -50
README.md CHANGED
@@ -37,28 +37,28 @@ InternVL 2.0 is a multimodal large language model series, featuring models of va
37
 
38
  ### Image Benchmarks
39
 
40
- | Benchmark | GPT-4o-20240513 | Claude3.5-Sonnet | InternVL2-40B | InternVL2-Llama3-76B |
41
- | :-----------------------------: | :-------------: | :--------------: | :-----------: | :------------------: |
42
- | Model Size | - | - | 40B | 76B |
43
- | | | | | |
44
- | DocVQA<sub>test</sub> | 92.8 | 95.2 | 93.9 | 94.1 |
45
- | ChartQA<sub>test</sub> | 85.7 | 90.8 | 86.2 | 88.4 |
46
- | InfoVQA<sub>test</sub> | - | - | 78.7 | 82.0 |
47
- | TextVQA<sub>val</sub> | - | - | 83.0 | 84.4 |
48
- | OCRBench | 736 | 788 | 837 | 839 |
49
- | MME<sub>sum</sub> | 2328.7 | 1920.0 | 2315.0 | 2414.7 |
50
- | RealWorldQA | 75.4 | 60.1 | 71.8 | 72.2 |
51
- | AI2D<sub>test</sub> | 94.2 | 94.7 | 87.1 | 87.6 |
52
- | MMMU<sub>val</sub> | 69.1 / 69.2 | 68.3 / 65.9 | 53.9 / 55.2 | 55.2 / 58.2 |
53
- | MMBench-EN<sub>test</sub> | 83.4 | 79.7 | 86.8 | 86.5 |
54
- | MMBench-CN<sub>test</sub> | 82.1 | 80.7 | 86.5 | 86.3 |
55
- | CCBench<sub>dev</sub> | 71.2 | 54.1 | 80.6 | 81.0 |
56
- | MMVet<sub>GPT-4-0613</sub> | - | - | 68.5 | 69.8 |
57
- | MMVet<sub>GPT-4-Turbo</sub> | 69.1 | 66.0 | 65.5 | 65.7 |
58
- | SEED-Image | 77.1 | - | 78.2 | 78.2 |
59
- | HallBench<sub>avg</sub> | 55.0 | 49.9 | 56.9 | 55.2 |
60
- | MathVista<sub>testmini</sub> | 63.8 | 67.7 | 63.7 | 65.5 |
61
- | OpenCompass<sub>avg-score</sub> | 69.9 | 67.9 | 69.7 | 70+ |
62
 
63
  - We simultaneously use InternVL and VLMEvalKit repositories for model evaluation. Specifically, the results reported for DocVQA, ChartQA, InfoVQA, TextVQA, MME, AI2D, MMBench, CCBench, MMVet, and SEED-Image were tested using the InternVL repository. OCRBench, RealWorldQA, HallBench, and MathVista were evaluated using the VLMEvalKit.
64
 
@@ -66,8 +66,6 @@ InternVL 2.0 is a multimodal large language model series, featuring models of va
66
 
67
  - Please note that evaluating the same model using different testing toolkits like InternVL and VLMEvalKit can result in slight differences, which is normal. Updates to code versions and variations in environment and hardware can also cause minor discrepancies in results.
68
 
69
- - It is important to mention that the MMVet scores we report are evaluated using GPT-4-0613 as the judge model. Different versions of GPT-4 can lead to significant variations in the scores for this dataset. For instance, using GPT-4-Turbo would result in significantly lower scores.
70
-
71
  ### Video Benchmarks
72
 
73
  | Benchmark | GPT-4V | VILA-1.5 | LLaVA-NeXT-Video | InternVL2-40B | InternVL2-Llama3-76B |
@@ -457,37 +455,35 @@ InternVL 2.0 是一个多模态大语言模型系列，包含各种规模的模
457
 
458
  ### 图像相关评测
459
 
460
- | 评测数据集 | GPT-4o-20240513 | Claude3.5-Sonnet | InternVL2-40B | InternVL2-Llama3-76B |
461
- | :-----------------------------: | :-------------: | :--------------: | :-----------: | :------------------: |
462
- | 模型大小 | - | - | 40B | 76B |
463
- | | | | | |
464
- | DocVQA<sub>test</sub> | 92.8 | 95.2 | 93.9 | 94.1 |
465
- | ChartQA<sub>test</sub> | 85.7 | 90.8 | 86.2 | 88.4 |
466
- | InfoVQA<sub>test</sub> | - | - | 78.7 | 82.0 |
467
- | TextVQA<sub>val</sub> | - | - | 83.0 | 84.4 |
468
- | OCRBench | 736 | 788 | 837 | 839 |
469
- | MME<sub>sum</sub> | 2328.7 | 1920.0 | 2315.0 | 2414.7 |
470
- | RealWorldQA | 75.4 | 60.1 | 71.8 | 72.2 |
471
- | AI2D<sub>test</sub> | 94.2 | 94.7 | 87.1 | 87.6 |
472
- | MMMU<sub>val</sub> | 69.1 / 69.2 | 68.3 / 65.9 | 53.9 / 55.2 | 55.2 / 58.2 |
473
- | MMBench-EN<sub>test</sub> | 83.4 | 79.7 | 86.8 | 86.5 |
474
- | MMBench-CN<sub>test</sub> | 82.1 | 80.7 | 86.5 | 86.3 |
475
- | CCBench<sub>dev</sub> | 71.2 | 54.1 | 80.6 | 81.0 |
476
- | MMVet<sub>GPT-4-0613</sub> | - | - | 68.5 | 69.8 |
477
- | MMVet<sub>GPT-4-Turbo</sub> | 69.1 | 66.0 | 65.5 | 65.7 |
478
- | SEED-Image | 77.1 | - | 78.2 | 78.2 |
479
- | HallBench<sub>avg</sub> | 55.0 | 49.9 | 56.9 | 55.2 |
480
- | MathVista<sub>testmini</sub> | 63.8 | 67.7 | 63.7 | 65.5 |
481
- | OpenCompass<sub>avg-score</sub> | 69.9 | 67.9 | 69.7 | 70+ |
482
-
483
- - 我们同时使用 InternVL 和 VLMEvalKit 仓库进行模型评估。具体来说，DocVQA、ChartQA、InfoVQA、TextVQA、MME、AI2D、MMBench、CCBench、MMVet 和 SEED-Image 的结果是使用 InternVL 仓库测试的。MMMU、OCRBench、RealWorldQA、HallBench 和 MathVista 是使用 VLMEvalKit 进行评估的。
484
 
485
  - 对于MMMU，我们报告了原始分数（左侧：InternVL系列模型使用InternVL代码库评测，其他模型的分数来自其技术报告或网页）和VLMEvalKit分数（右侧：从OpenCompass排行榜收集）。
486
 
487
  - 请注意，使用不同的测试工具包（如 InternVL 和 VLMEvalKit）评估同一模型可能会导致细微差异，这是正常的。代码版本的更新、环境和硬件的变化也可能导致结果的微小差异。
488
 
489
- - 需要提到的是，我们报告的 MMVet 分数是使用 GPT-4-0613 作为评判模型评估的。不同版本的 GPT-4 会导致该数据集分数的显著变化。例如，使用 GPT-4-Turbo 会导致分数显著降低。
490
-
491
  ### 视频相关评测
492
 
493
  | 评测数据集 | GPT-4V | VILA-1.5 | LLaVA-NeXT-Video | InternVL2-40B | InternVL2-Llama3-76B |
 
37
 
38
  ### Image Benchmarks
39
 
40
+ | Benchmark | GPT-4o-20240513 | Claude3.5-Sonnet | InternVL2-40B | InternVL2-Llama3-76B |
41
+ | :--------------------------: | :-------------: | :--------------: | :-----------: | :------------------: |
42
+ | Model Size | - | - | 40B | 76B |
43
+ | | | | | |
44
+ | DocVQA<sub>test</sub> | 92.8 | 95.2 | 93.9 | 94.1 |
45
+ | ChartQA<sub>test</sub> | 85.7 | 90.8 | 86.2 | 88.4 |
46
+ | InfoVQA<sub>test</sub> | - | - | 78.7 | 82.0 |
47
+ | TextVQA<sub>val</sub> | - | - | 83.0 | 84.4 |
48
+ | OCRBench | 736 | 788 | 837 | 839 |
49
+ | MME<sub>sum</sub> | 2328.7 | 1920.0 | 2315.0 | 2414.7 |
50
+ | RealWorldQA | 75.4 | 60.1 | 71.8 | 72.2 |
51
+ | AI2D<sub>test</sub> | 94.2 | 94.7 | 87.1 | 87.6 |
52
+ | MMMU<sub>val</sub> | 69.1 / 69.2 | 68.3 / 65.9 | 53.9 / 55.2 | 55.2 / 58.2 |
53
+ | MMBench-EN<sub>test</sub> | 83.4 | 79.7 | 86.8 | 86.5 |
54
+ | MMBench-CN<sub>test</sub> | 82.1 | 80.7 | 86.5 | 86.3 |
55
+ | CCBench<sub>dev</sub> | 71.2 | 54.1 | 80.6 | 81.0 |
56
+ | MMVet<sub>GPT-4-0613</sub> | - | - | 68.5 | 69.8 |
57
+ | MMVet<sub>GPT-4-Turbo</sub> | 69.1 | 66.0 | 65.5 | 65.7 |
58
+ | SEED-Image | 77.1 | - | 78.2 | 78.2 |
59
+ | HallBench<sub>avg</sub> | 55.0 | 49.9 | 56.9 | 55.2 |
60
+ | MathVista<sub>testmini</sub> | 63.8 | 67.7 | 63.7 | 65.5 |
61
+ | OpenCompass<sub>avg</sub> | 69.9 | 67.9 | 69.7 | 70+ |
62
 
63
  - We simultaneously use InternVL and VLMEvalKit repositories for model evaluation. Specifically, the results reported for DocVQA, ChartQA, InfoVQA, TextVQA, MME, AI2D, MMBench, CCBench, MMVet, and SEED-Image were tested using the InternVL repository. OCRBench, RealWorldQA, HallBench, and MathVista were evaluated using the VLMEvalKit.
64
 
 
66
 
67
  - Please note that evaluating the same model using different testing toolkits like InternVL and VLMEvalKit can result in slight differences, which is normal. Updates to code versions and variations in environment and hardware can also cause minor discrepancies in results.
68
 
 
 
69
  ### Video Benchmarks
70
 
71
  | Benchmark | GPT-4V | VILA-1.5 | LLaVA-NeXT-Video | InternVL2-40B | InternVL2-Llama3-76B |
 
455
 
456
  ### 图像相关评测
457
 
458
+ | 评测数据集 | GPT-4o-20240513 | Claude3.5-Sonnet | InternVL2-40B | InternVL2-Llama3-76B |
459
+ | :--------------------------: | :-------------: | :--------------: | :-----------: | :------------------: |
460
+ | 模型大小 | - | - | 40B | 76B |
461
+ | | | | | |
462
+ | DocVQA<sub>test</sub> | 92.8 | 95.2 | 93.9 | 94.1 |
463
+ | ChartQA<sub>test</sub> | 85.7 | 90.8 | 86.2 | 88.4 |
464
+ | InfoVQA<sub>test</sub> | - | - | 78.7 | 82.0 |
465
+ | TextVQA<sub>val</sub> | - | - | 83.0 | 84.4 |
466
+ | OCRBench | 736 | 788 | 837 | 839 |
467
+ | MME<sub>sum</sub> | 2328.7 | 1920.0 | 2315.0 | 2414.7 |
468
+ | RealWorldQA | 75.4 | 60.1 | 71.8 | 72.2 |
469
+ | AI2D<sub>test</sub> | 94.2 | 94.7 | 87.1 | 87.6 |
470
+ | MMMU<sub>val</sub> | 69.1 / 69.2 | 68.3 / 65.9 | 53.9 / 55.2 | 55.2 / 58.2 |
471
+ | MMBench-EN<sub>test</sub> | 83.4 | 79.7 | 86.8 | 86.5 |
472
+ | MMBench-CN<sub>test</sub> | 82.1 | 80.7 | 86.5 | 86.3 |
473
+ | CCBench<sub>dev</sub> | 71.2 | 54.1 | 80.6 | 81.0 |
474
+ | MMVet<sub>GPT-4-0613</sub> | - | - | 68.5 | 69.8 |
475
+ | MMVet<sub>GPT-4-Turbo</sub> | 69.1 | 66.0 | 65.5 | 65.7 |
476
+ | SEED-Image | 77.1 | - | 78.2 | 78.2 |
477
+ | HallBench<sub>avg</sub> | 55.0 | 49.9 | 56.9 | 55.2 |
478
+ | MathVista<sub>testmini</sub> | 63.8 | 67.7 | 63.7 | 65.5 |
479
+ | OpenCompass<sub>avg</sub> | 69.9 | 67.9 | 69.7 | 70+ |
480
+
481
+ - 我们同时使用 InternVL 和 VLMEvalKit 仓库进行模型评估。具体来说，DocVQA、ChartQA、InfoVQA、TextVQA、MME、AI2D、MMBench、CCBench、MMVet 和 SEED-Image 的结果是使用 InternVL 仓库测试的。OCRBench、RealWorldQA、HallBench 和 MathVista 是使用 VLMEvalKit 进行评估的。
482
 
483
  - 对于MMMU，我们报告了原始分数（左侧：InternVL系列模型使用InternVL代码库评测，其他模型的分数来自其技术报告或网页）和VLMEvalKit分数（右侧：从OpenCompass排行榜收集）。
484
 
485
  - 请注意，使用不同的测试工具包（如 InternVL 和 VLMEvalKit）评估同一模型可能会导致细微差异，这是正常的。代码版本的更新、环境和硬件的变化也可能导致结果的微小差异。
486
 
 
 
487
  ### 视频相关评测
488
 
489
  | 评测数据集 | GPT-4V | VILA-1.5 | LLaVA-NeXT-Video | InternVL2-40B | InternVL2-Llama3-76B |