Hugging Face's logo

Snowkcon
/
bge_large_cn_new_prompt_llama3_70

Safetensors
bert
Generated from Trainer
Model card Files Community
Snowkcon commited on
Commit
9962677
·
verified ·
1 Parent(s): 4296849

Model save

Browse files
Files changed (9) hide show
  1. README.md +117 -0
  2. final/config.json +41 -0
  3. final/model.safetensors +3 -0
  4. final/special_tokens_map.json +37 -0
  5. final/tokenizer.json +0 -0
  6. final/tokenizer_config.json +57 -0
  7. final/training_args.bin +3 -0
  8. final/vocab.txt +0 -0
  9. model.safetensors +1 -1
README.md ADDED
@@ -0,0 +1,117 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - generated_from_trainer
4
+ metrics:
5
+ - precision
6
+ - recall
7
+ - accuracy
8
+ model-index:
9
+ - name: bge_large_cn_new_prompt_llama3_70
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ # bge_large_cn_new_prompt_llama3_70
17
+
18
+ This model was trained from scratch on an unknown dataset.
19
+ It achieves the following results on the evaluation set:
20
+ - Loss: 0.9517
21
+ - Precision: 0.4483
22
+ - Recall: 0.3656
23
+ - F1 Macro: 0.3577
24
+ - Accuracy: 0.4617
25
+
26
+ ## Model description
27
+
28
+ More information needed
29
+
30
+ ## Intended uses & limitations
31
+
32
+ More information needed
33
+
34
+ ## Training and evaluation data
35
+
36
+ More information needed
37
+
38
+ ## Training procedure
39
+
40
+ ### Training hyperparameters
41
+
42
+ The following hyperparameters were used during training:
43
+ - learning_rate: 0.0003
44
+ - train_batch_size: 256
45
+ - eval_batch_size: 128
46
+ - seed: 0
47
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
+ - lr_scheduler_type: linear
49
+ - num_epochs: 20
50
+
51
+ ### Training results
52
+
53
+ | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 Macro | Accuracy |
54
+ |:-------------:|:-------:|:-----:|:---------------:|:---------:|:------:|:--------:|:--------:|
55
+ | No log | 0 | 0 | 10.4077 | 0.1217 | 0.1667 | 0.0201 | 0.0638 |
56
+ | 1.01 | 0.3672 | 1000 | 1.0195 | 0.4334 | 0.3321 | 0.3218 | 0.4338 |
57
+ | 1.0013 | 0.7345 | 2000 | 0.9928 | 0.4524 | 0.3418 | 0.3300 | 0.4429 |
58
+ | 0.9922 | 1.1017 | 3000 | 0.9758 | 0.4591 | 0.3556 | 0.3460 | 0.4574 |
59
+ | 0.9353 | 1.4690 | 4000 | 0.9716 | 0.4607 | 0.3557 | 0.3431 | 0.4597 |
60
+ | 0.9501 | 1.8362 | 5000 | 0.9638 | 0.4765 | 0.3546 | 0.3402 | 0.4627 |
61
+ | 0.9574 | 2.2035 | 6000 | 0.9628 | 0.4630 | 0.3502 | 0.3320 | 0.4527 |
62
+ | 0.9324 | 2.5707 | 7000 | 0.9755 | 0.4565 | 0.3592 | 0.3448 | 0.4471 |
63
+ | 0.9373 | 2.9379 | 8000 | 0.9504 | 0.4749 | 0.3635 | 0.3503 | 0.4694 |
64
+ | 0.9303 | 3.3052 | 9000 | 0.9510 | 0.4809 | 0.3638 | 0.3516 | 0.4702 |
65
+ | 0.9453 | 3.6724 | 10000 | 0.9519 | 0.4622 | 0.3619 | 0.3472 | 0.4588 |
66
+ | 0.9278 | 4.0397 | 11000 | 0.9469 | 0.4678 | 0.3604 | 0.3453 | 0.4591 |
67
+ | 0.9246 | 4.4069 | 12000 | 0.9440 | 0.4731 | 0.3634 | 0.3476 | 0.4628 |
68
+ | 0.9229 | 4.7741 | 13000 | 0.9423 | 0.4778 | 0.3666 | 0.3539 | 0.4704 |
69
+ | 0.9105 | 5.1414 | 14000 | 0.9388 | 0.4717 | 0.3655 | 0.3511 | 0.4647 |
70
+ | 0.8994 | 5.5086 | 15000 | 0.9471 | 0.4807 | 0.3638 | 0.3517 | 0.4707 |
71
+ | 0.9138 | 5.8759 | 16000 | 0.9412 | 0.4768 | 0.3620 | 0.3489 | 0.4692 |
72
+ | 0.9157 | 6.2431 | 17000 | 0.9393 | 0.4703 | 0.3638 | 0.3495 | 0.4631 |
73
+ | 0.9074 | 6.6104 | 18000 | 0.9372 | 0.4699 | 0.3676 | 0.3553 | 0.4679 |
74
+ | 0.9048 | 6.9776 | 19000 | 0.9378 | 0.4671 | 0.3688 | 0.3575 | 0.4690 |
75
+ | 0.8827 | 7.3448 | 20000 | 0.9421 | 0.4603 | 0.3673 | 0.3550 | 0.4628 |
76
+ | 0.888 | 7.7121 | 21000 | 0.9384 | 0.4632 | 0.3675 | 0.3584 | 0.4655 |
77
+ | 0.8701 | 8.0793 | 22000 | 0.9449 | 0.4617 | 0.3665 | 0.3535 | 0.4614 |
78
+ | 0.8808 | 8.4466 | 23000 | 0.9382 | 0.4595 | 0.3642 | 0.3517 | 0.4668 |
79
+ | 0.901 | 8.8138 | 24000 | 0.9436 | 0.4693 | 0.3654 | 0.3554 | 0.4682 |
80
+ | 0.8729 | 9.1811 | 25000 | 0.9459 | 0.4624 | 0.3625 | 0.3483 | 0.4580 |
81
+ | 0.8798 | 9.5483 | 26000 | 0.9459 | 0.4546 | 0.3665 | 0.3541 | 0.4581 |
82
+ | 0.8745 | 9.9155 | 27000 | 0.9518 | 0.4633 | 0.3688 | 0.3623 | 0.4700 |
83
+ | 0.8484 | 10.2828 | 28000 | 0.9426 | 0.4568 | 0.3655 | 0.3543 | 0.4623 |
84
+ | 0.8654 | 10.6500 | 29000 | 0.9448 | 0.4689 | 0.3659 | 0.3560 | 0.4680 |
85
+ | 0.8543 | 11.0173 | 30000 | 0.9421 | 0.4614 | 0.3625 | 0.3507 | 0.4645 |
86
+ | 0.8446 | 11.3845 | 31000 | 0.9451 | 0.4600 | 0.3684 | 0.3606 | 0.4671 |
87
+ | 0.8589 | 11.7517 | 32000 | 0.9414 | 0.4597 | 0.3672 | 0.3578 | 0.4644 |
88
+ | 0.8201 | 12.1190 | 33000 | 0.9469 | 0.4568 | 0.3641 | 0.3533 | 0.4587 |
89
+ | 0.8303 | 12.4862 | 34000 | 0.9527 | 0.4603 | 0.3652 | 0.3578 | 0.4663 |
90
+ | 0.8391 | 12.8535 | 35000 | 0.9466 | 0.4568 | 0.3647 | 0.3532 | 0.4601 |
91
+ | 0.8393 | 13.2207 | 36000 | 0.9489 | 0.4518 | 0.3612 | 0.3488 | 0.4555 |
92
+ | 0.8388 | 13.5880 | 37000 | 0.9463 | 0.4575 | 0.3639 | 0.3536 | 0.4628 |
93
+ | 0.8396 | 13.9552 | 38000 | 0.9466 | 0.4554 | 0.3663 | 0.3557 | 0.4635 |
94
+ | 0.8296 | 14.3224 | 39000 | 0.9488 | 0.4555 | 0.3665 | 0.3575 | 0.4661 |
95
+ | 0.8351 | 14.6897 | 40000 | 0.9500 | 0.4523 | 0.3675 | 0.3610 | 0.4635 |
96
+ | 0.8362 | 15.0569 | 41000 | 0.9500 | 0.4585 | 0.3645 | 0.3559 | 0.4637 |
97
+ | 0.8174 | 15.4242 | 42000 | 0.9510 | 0.4547 | 0.3700 | 0.3634 | 0.4646 |
98
+ | 0.8249 | 15.7914 | 43000 | 0.9541 | 0.4513 | 0.3670 | 0.3605 | 0.4642 |
99
+ | 0.8057 | 16.1586 | 44000 | 0.9504 | 0.4519 | 0.3646 | 0.3562 | 0.4604 |
100
+ | 0.8145 | 16.5259 | 45000 | 0.9539 | 0.4537 | 0.3667 | 0.3595 | 0.4641 |
101
+ | 0.8237 | 16.8931 | 46000 | 0.9515 | 0.4524 | 0.3662 | 0.3582 | 0.4622 |
102
+ | 0.8082 | 17.2604 | 47000 | 0.9515 | 0.4484 | 0.3646 | 0.3547 | 0.4588 |
103
+ | 0.8249 | 17.6276 | 48000 | 0.9512 | 0.4488 | 0.3651 | 0.3573 | 0.4608 |
104
+ | 0.8074 | 17.9949 | 49000 | 0.9507 | 0.4498 | 0.3662 | 0.3584 | 0.4610 |
105
+ | 0.7944 | 18.3621 | 50000 | 0.9512 | 0.4516 | 0.3649 | 0.3554 | 0.4605 |
106
+ | 0.8041 | 18.7293 | 51000 | 0.9520 | 0.4515 | 0.3660 | 0.3581 | 0.4623 |
107
+ | 0.7958 | 19.0966 | 52000 | 0.9528 | 0.4497 | 0.3651 | 0.3574 | 0.4617 |
108
+ | 0.8173 | 19.4638 | 53000 | 0.9518 | 0.4511 | 0.3644 | 0.3558 | 0.4615 |
109
+ | 0.7909 | 19.8311 | 54000 | 0.9517 | 0.4483 | 0.3656 | 0.3577 | 0.4617 |
110
+
111
+
112
+ ### Framework versions
113
+
114
+ - Transformers 4.43.3
115
+ - Pytorch 2.4.0+cu121
116
+ - Datasets 2.20.0
117
+ - Tokenizers 0.19.1
final/config.json ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "/mnt/workspace/models/bge-large-zh-v1.5",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "classifier_dropout": 0.0,
9
+ "directionality": "bidi",
10
+ "eos_token_id": 2,
11
+ "hidden_act": "gelu",
12
+ "hidden_dropout_prob": 0.0,
13
+ "hidden_size": 1024,
14
+ "id2label": {
15
+ "0": "LABEL_0"
16
+ },
17
+ "initializer_range": 0.02,
18
+ "intermediate_size": 4096,
19
+ "label2id": {
20
+ "LABEL_0": 0
21
+ },
22
+ "layer_norm_eps": 1e-12,
23
+ "max_position_embeddings": 512,
24
+ "model_type": "bert",
25
+ "num_attention_heads": 16,
26
+ "num_hidden_layers": 24,
27
+ "output_past": true,
28
+ "pad_token_id": 0,
29
+ "pooler_fc_size": 768,
30
+ "pooler_num_attention_heads": 12,
31
+ "pooler_num_fc_layers": 3,
32
+ "pooler_size_per_head": 128,
33
+ "pooler_type": "first_token_transform",
34
+ "position_embedding_type": "absolute",
35
+ "problem_type": "regression",
36
+ "torch_dtype": "float32",
37
+ "transformers_version": "4.43.3",
38
+ "type_vocab_size": 2,
39
+ "use_cache": true,
40
+ "vocab_size": 21128
41
+ }
final/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c6918fbda3b1ab7169c2627ae6f5ab47b32160f329b367ace247dd52951ebc0
3
+ size 1302140796
final/special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
final/tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
final/tokenizer_config.json ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": true,
45
+ "cls_token": "[CLS]",
46
+ "do_basic_tokenize": true,
47
+ "do_lower_case": true,
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "never_split": null,
51
+ "pad_token": "[PAD]",
52
+ "sep_token": "[SEP]",
53
+ "strip_accents": null,
54
+ "tokenize_chinese_chars": true,
55
+ "tokenizer_class": "BertTokenizer",
56
+ "unk_token": "[UNK]"
57
+ }
final/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:161860f753731806e1413c550e385033c10bf36d5fe475b924f2517bbd61bdbf
3
+ size 5304
final/vocab.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:66cdec35578ff692d3ed238ab551a94d78dbe6a90ce0a9a8ec43637167f62130
3
  size 1302140796
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1c6918fbda3b1ab7169c2627ae6f5ab47b32160f329b367ace247dd52951ebc0
3
  size 1302140796