Hugging Face's logo

thelikhit
/
HuBERT-draft-version

Audio Classification
Transformers
TensorBoard
Safetensors
hubert
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Metrics Community
thelikhit commited on
Commit
40d9c86
•
1 Parent(s): 2be3c93

End of training

Browse files
Files changed (14) hide show
  1. README.md +97 -0
  2. config.json +100 -0
  3. model.safetensors +3 -0
  4. preprocessor_config.json +9 -0
  5. runs/Dec03_13-48-25_d63287f5e78f/events.out.tfevents.1733233707.d63287f5e78f.962.0 +3 -0
  6. runs/Dec03_13-55-22_d63287f5e78f/events.out.tfevents.1733234123.d63287f5e78f.962.1 +3 -0
  7. runs/Dec03_13-56-05_d63287f5e78f/events.out.tfevents.1733234166.d63287f5e78f.962.2 +3 -0
  8. runs/Dec03_13-57-51_d63287f5e78f/events.out.tfevents.1733234271.d63287f5e78f.962.3 +3 -0
  9. runs/Dec03_13-58-48_d63287f5e78f/events.out.tfevents.1733234328.d63287f5e78f.962.4 +3 -0
  10. runs/Dec03_13-59-23_d63287f5e78f/events.out.tfevents.1733234364.d63287f5e78f.962.5 +3 -0
  11. runs/Dec03_14-00-43_d63287f5e78f/events.out.tfevents.1733234444.d63287f5e78f.962.6 +3 -0
  12. runs/Dec03_14-01-13_d63287f5e78f/events.out.tfevents.1733234474.d63287f5e78f.962.7 +3 -0
  13. runs/Dec03_14-06-01_d63287f5e78f/events.out.tfevents.1733234761.d63287f5e78f.962.8 +3 -0
  14. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,97 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: facebook/hubert-base-ls960
5
+ tags:
6
+ - generated_from_trainer
7
+ datasets:
8
+ - gtzan
9
+ metrics:
10
+ - accuracy
11
+ model-index:
12
+ - name: hubert-test-model
13
+ results:
14
+ - task:
15
+ name: Audio Classification
16
+ type: audio-classification
17
+ dataset:
18
+ name: gtzan
19
+ type: gtzan
20
+ config: default
21
+ split: train
22
+ args: default
23
+ metrics:
24
+ - name: Accuracy
25
+ type: accuracy
26
+ value: 0.785
27
+ ---
28
+
29
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
30
+ should probably proofread and complete it, then remove this comment. -->
31
+
32
+ # hubert-test-model
33
+
34
+ This model is a fine-tuned version of [facebook/hubert-base-ls960](https://huggingface.co/facebook/hubert-base-ls960) on the gtzan dataset.
35
+ It achieves the following results on the evaluation set:
36
+ - Loss: 1.5276
37
+ - Accuracy: 0.785
38
+
39
+ ## Model description
40
+
41
+ More information needed
42
+
43
+ ## Intended uses & limitations
44
+
45
+ More information needed
46
+
47
+ ## Training and evaluation data
48
+
49
+ More information needed
50
+
51
+ ## Training procedure
52
+
53
+ ### Training hyperparameters
54
+
55
+ The following hyperparameters were used during training:
56
+ - learning_rate: 5e-05
57
+ - train_batch_size: 4
58
+ - eval_batch_size: 4
59
+ - seed: 42
60
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
61
+ - lr_scheduler_type: linear
62
+ - lr_scheduler_warmup_ratio: 0.1
63
+ - num_epochs: 20
64
+ - mixed_precision_training: Native AMP
65
+
66
+ ### Training results
67
+
68
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
69
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|
70
+ | No log | 1.0 | 150 | 2.0494 | 0.29 |
71
+ | No log | 2.0 | 300 | 2.1993 | 0.19 |
72
+ | No log | 3.0 | 450 | 1.8439 | 0.44 |
73
+ | 1.9218 | 4.0 | 600 | 1.5277 | 0.48 |
74
+ | 1.9218 | 5.0 | 750 | 1.4164 | 0.475 |
75
+ | 1.9218 | 6.0 | 900 | 1.3641 | 0.63 |
76
+ | 1.2685 | 7.0 | 1050 | 1.1557 | 0.675 |
77
+ | 1.2685 | 8.0 | 1200 | 1.0935 | 0.72 |
78
+ | 1.2685 | 9.0 | 1350 | 1.0594 | 0.71 |
79
+ | 0.7151 | 10.0 | 1500 | 1.0119 | 0.735 |
80
+ | 0.7151 | 11.0 | 1650 | 1.0868 | 0.77 |
81
+ | 0.7151 | 12.0 | 1800 | 1.3736 | 0.75 |
82
+ | 0.7151 | 13.0 | 1950 | 1.2705 | 0.77 |
83
+ | 0.4135 | 14.0 | 2100 | 1.4052 | 0.76 |
84
+ | 0.4135 | 15.0 | 2250 | 1.3864 | 0.77 |
85
+ | 0.4135 | 16.0 | 2400 | 1.4296 | 0.785 |
86
+ | 0.2311 | 17.0 | 2550 | 1.5663 | 0.77 |
87
+ | 0.2311 | 18.0 | 2700 | 1.5310 | 0.78 |
88
+ | 0.2311 | 19.0 | 2850 | 1.4884 | 0.795 |
89
+ | 0.1408 | 20.0 | 3000 | 1.5276 | 0.785 |
90
+
91
+
92
+ ### Framework versions
93
+
94
+ - Transformers 4.46.2
95
+ - Pytorch 2.5.1+cu121
96
+ - Datasets 3.1.0
97
+ - Tokenizers 0.20.3
config.json ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "facebook/hubert-base-ls960",
3
+ "activation_dropout": 0.1,
4
+ "apply_spec_augment": true,
5
+ "architectures": [
6
+ "HubertForSequenceClassification"
7
+ ],
8
+ "attention_dropout": 0.1,
9
+ "bos_token_id": 1,
10
+ "classifier_proj_size": 256,
11
+ "conv_bias": false,
12
+ "conv_dim": [
13
+ 512,
14
+ 512,
15
+ 512,
16
+ 512,
17
+ 512,
18
+ 512,
19
+ 512
20
+ ],
21
+ "conv_kernel": [
22
+ 10,
23
+ 3,
24
+ 3,
25
+ 3,
26
+ 3,
27
+ 2,
28
+ 2
29
+ ],
30
+ "conv_stride": [
31
+ 5,
32
+ 2,
33
+ 2,
34
+ 2,
35
+ 2,
36
+ 2,
37
+ 2
38
+ ],
39
+ "ctc_loss_reduction": "sum",
40
+ "ctc_zero_infinity": false,
41
+ "do_stable_layer_norm": false,
42
+ "eos_token_id": 2,
43
+ "feat_extract_activation": "gelu",
44
+ "feat_extract_dropout": 0.0,
45
+ "feat_extract_norm": "group",
46
+ "feat_proj_dropout": 0.1,
47
+ "feat_proj_layer_norm": true,
48
+ "final_dropout": 0.1,
49
+ "gradient_checkpointing": false,
50
+ "hidden_act": "gelu",
51
+ "hidden_dropout": 0.1,
52
+ "hidden_dropout_prob": 0.1,
53
+ "hidden_size": 768,
54
+ "id2label": {
55
+ "0": "blues",
56
+ "1": "classical",
57
+ "2": "country",
58
+ "3": "disco",
59
+ "4": "hiphop",
60
+ "5": "jazz",
61
+ "6": "metal",
62
+ "7": "pop",
63
+ "8": "reggae",
64
+ "9": "rock"
65
+ },
66
+ "initializer_range": 0.02,
67
+ "intermediate_size": 3072,
68
+ "label2id": {
69
+ "blues": "0",
70
+ "classical": "1",
71
+ "country": "2",
72
+ "disco": "3",
73
+ "hiphop": "4",
74
+ "jazz": "5",
75
+ "metal": "6",
76
+ "pop": "7",
77
+ "reggae": "8",
78
+ "rock": "9"
79
+ },
80
+ "layer_norm_eps": 1e-05,
81
+ "layerdrop": 0.1,
82
+ "mask_feature_length": 10,
83
+ "mask_feature_min_masks": 0,
84
+ "mask_feature_prob": 0.0,
85
+ "mask_time_length": 10,
86
+ "mask_time_min_masks": 2,
87
+ "mask_time_prob": 0.05,
88
+ "model_type": "hubert",
89
+ "num_attention_heads": 12,
90
+ "num_conv_pos_embedding_groups": 16,
91
+ "num_conv_pos_embeddings": 128,
92
+ "num_feat_extract_layers": 7,
93
+ "num_hidden_layers": 12,
94
+ "pad_token_id": 0,
95
+ "tokenizer_class": "Wav2Vec2CTCTokenizer",
96
+ "torch_dtype": "float32",
97
+ "transformers_version": "4.46.2",
98
+ "use_weighted_layer_sum": false,
99
+ "vocab_size": 32
100
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e399da03054e084360b1f6cbde1fd49ff43371b932baac123137d19b2be3695
3
+ size 378310176
preprocessor_config.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "do_normalize": true,
3
+ "feature_extractor_type": "Wav2Vec2FeatureExtractor",
4
+ "feature_size": 1,
5
+ "padding_side": "right",
6
+ "padding_value": 0,
7
+ "return_attention_mask": true,
8
+ "sampling_rate": 16000
9
+ }
runs/Dec03_13-48-25_d63287f5e78f/events.out.tfevents.1733233707.d63287f5e78f.962.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c5348dc1aefaa96f7cfa49c59af3aa5e6086506421f9bf95841fb8e87e40e4e2
3
+ size 6358
runs/Dec03_13-55-22_d63287f5e78f/events.out.tfevents.1733234123.d63287f5e78f.962.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b095634643c1555c547526bbfc0fa7adc5f8f48715558f30776e6f432c1e9dd
3
+ size 6357
runs/Dec03_13-56-05_d63287f5e78f/events.out.tfevents.1733234166.d63287f5e78f.962.2 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fd197d65767ad47d49455469738a0ab32718deaba50b2c3bd8b6ae01123dad44
3
+ size 6357
runs/Dec03_13-57-51_d63287f5e78f/events.out.tfevents.1733234271.d63287f5e78f.962.3 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1fca96f9b2d61130d994d071973af6af72e448370afa80d4c4073fc9c32d22ce
3
+ size 6357
runs/Dec03_13-58-48_d63287f5e78f/events.out.tfevents.1733234328.d63287f5e78f.962.4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4479c8f2ad76d925ed11a4b2cc83143e11f40c66dcd028c291ea1200676c6e4a
3
+ size 6357
runs/Dec03_13-59-23_d63287f5e78f/events.out.tfevents.1733234364.d63287f5e78f.962.5 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb309fc1ce7996b4fc96b5d5ad24ab3a689b418cd20c2a95738ae8a4d1e16754
3
+ size 6357
runs/Dec03_14-00-43_d63287f5e78f/events.out.tfevents.1733234444.d63287f5e78f.962.6 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86778ffc04c91f6050301b8390667f7576904fc4d8aabc4dde9f5223a2de70f5
3
+ size 6357
runs/Dec03_14-01-13_d63287f5e78f/events.out.tfevents.1733234474.d63287f5e78f.962.7 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:687f060c81203b53e8b6dd23907507a7358c7c59a3759cbae0abdf29f13cc2f2
3
+ size 6357
runs/Dec03_14-06-01_d63287f5e78f/events.out.tfevents.1733234761.d63287f5e78f.962.8 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91e350ac79fd24c3139ee9f4a158aaf68b30027702391693bddb87054f697112
3
+ size 14437
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3ce16e75c6197b52fd2ac42f4f8f899f3d42cdb2a7017f291b2b702bbf81b205
3
+ size 5304