--- library_name: transformers tags: - Spanish - generated_from_trainer model-index: - name: childes-multilingual-5M-gpt2_lm-model results: [] --- # childes-multilingual-5M-gpt2_lm-model This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 1.6555 - Model Preparation Time: 0.0013 - Perplexity: 5.2355 - Bpc: 2.3883 - Spike Seg Type Fscore Entropy: 0.3332 - Spike Seg Boundary Fscore Entropy: 0.6727 - Absolute Seg Type Fscore Entropy: 0.3995 - Absolute Seg Boundary Fscore Entropy: 0.6660 - Spike Seg Type Fscore Increase in entropy: 0.2810 - Spike Seg Boundary Fscore Increase in entropy: 0.6403 - Absolute Seg Type Fscore Increase in entropy: 0.3995 - Absolute Seg Boundary Fscore Increase in entropy: 0.6829 - Spike Seg Type Fscore Loss: 0.2547 - Spike Seg Boundary Fscore Loss: 0.5907 - Absolute Seg Type Fscore Loss: 0.2617 - Absolute Seg Boundary Fscore Loss: 0.5870 - Spike Seg Type Fscore Increase in loss: 0.2604 - Spike Seg Boundary Fscore Increase in loss: 0.6271 - Absolute Seg Type Fscore Increase in loss: 0.3425 - Absolute Seg Boundary Fscore Increase in loss: 0.6148 - Spike Seg Type Fscore Rank: 0.3266 - Spike Seg Boundary Fscore Rank: 0.6156 - Absolute Seg Type Fscore Rank: 0.2731 - Absolute Seg Boundary Fscore Rank: 0.5804 - Spike Seg Type Fscore Increase in rank: 0.3286 - Spike Seg Boundary Fscore Increase in rank: 0.6367 - Absolute Seg Type Fscore Increase in rank: 0.3916 - Absolute Seg Boundary Fscore Increase in rank: 0.6459 - Spike Seg Type Fscore Boundary prediction: 0.2640 - Spike Seg Boundary Fscore Boundary prediction: 0.6591 - Absolute Seg Type Fscore Boundary prediction: 0.4087 - Absolute Seg Boundary Fscore Boundary prediction: 0.6786 - Spike Seg Type Fscore Increase in boundary prediction: 0.2521 - Spike Seg Boundary Fscore Increase in boundary prediction: 0.6460 - Absolute Seg Type Fscore Increase in boundary prediction: 0.3788 - Absolute Seg Boundary Fscore Increase in boundary prediction: 0.6863 - Spike Seg Type Fscore Majority vote cutoff: 0.4145 - Spike Seg Type Fscore Majority vote spike: 0.2634 - Absolute Seg Type Fscore Majority vote cutoff: 0.4730 - Absolute Seg Type Fscore Majority vote spike: 0.3846 - Spike Seg Boundary Fscore Majority vote cutoff: 0.7263 - Spike Seg Boundary Fscore Majority vote spike: 0.6556 - Absolute Seg Boundary Fscore Majority vote cutoff: 0.7257 - Absolute Seg Boundary Fscore Majority vote spike: 0.7086 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.001 - train_batch_size: 32 - eval_batch_size: 32 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 60000 - training_steps: 200000 ### Training results | Training Loss | Epoch | Step | Validation Loss | Model Preparation Time | Perplexity | Bpc | Spike Seg Type Fscore Entropy | Spike Seg Boundary Fscore Entropy | Absolute Seg Type Fscore Entropy | Absolute Seg Boundary Fscore Entropy | Spike Seg Type Fscore Increase in entropy | Spike Seg Boundary Fscore Increase in entropy | Absolute Seg Type Fscore Increase in entropy | Absolute Seg Boundary Fscore Increase in entropy | Spike Seg Type Fscore Loss | Spike Seg Boundary Fscore Loss | Absolute Seg Type Fscore Loss | Absolute Seg Boundary Fscore Loss | Spike Seg Type Fscore Increase in loss | Spike Seg Boundary Fscore Increase in loss | Absolute Seg Type Fscore Increase in loss | Absolute Seg Boundary Fscore Increase in loss | Spike Seg Type Fscore Rank | Spike Seg Boundary Fscore Rank | Absolute Seg Type Fscore Rank | Absolute Seg Boundary Fscore Rank | Spike Seg Type Fscore Increase in rank | Spike Seg Boundary Fscore Increase in rank | Absolute Seg Type Fscore Increase in rank | Absolute Seg Boundary Fscore Increase in rank | Spike Seg Type Fscore Boundary prediction | Spike Seg Boundary Fscore Boundary prediction | Absolute Seg Type Fscore Boundary prediction | Absolute Seg Boundary Fscore Boundary prediction | Spike Seg Type Fscore Increase in boundary prediction | Spike Seg Boundary Fscore Increase in boundary prediction | Absolute Seg Type Fscore Increase in boundary prediction | Absolute Seg Boundary Fscore Increase in boundary prediction | Spike Seg Type Fscore Majority vote cutoff | Spike Seg Type Fscore Majority vote spike | Absolute Seg Type Fscore Majority vote cutoff | Absolute Seg Type Fscore Majority vote spike | Spike Seg Boundary Fscore Majority vote cutoff | Spike Seg Boundary Fscore Majority vote spike | Absolute Seg Boundary Fscore Majority vote cutoff | Absolute Seg Boundary Fscore Majority vote spike | |:-------------:|:--------:|:------:|:---------------:|:----------------------:|:----------:|:------:|:-----------------------------:|:---------------------------------:|:--------------------------------:|:------------------------------------:|:-----------------------------------------:|:---------------------------------------------:|:--------------------------------------------:|:------------------------------------------------:|:--------------------------:|:------------------------------:|:-----------------------------:|:---------------------------------:|:--------------------------------------:|:------------------------------------------:|:-----------------------------------------:|:---------------------------------------------:|:--------------------------:|:------------------------------:|:-----------------------------:|:---------------------------------:|:--------------------------------------:|:------------------------------------------:|:-----------------------------------------:|:---------------------------------------------:|:-----------------------------------------:|:---------------------------------------------:|:--------------------------------------------:|:------------------------------------------------:|:-----------------------------------------------------:|:---------------------------------------------------------:|:--------------------------------------------------------:|:------------------------------------------------------------:|:------------------------------------------:|:-----------------------------------------:|:---------------------------------------------:|:--------------------------------------------:|:----------------------------------------------:|:---------------------------------------------:|:-------------------------------------------------:|:------------------------------------------------:| | 1.4761 | 45.3515 | 20000 | 1.5296 | 0.0013 | 4.6161 | 2.2067 | 0.2933 | 0.6533 | 0.3764 | 0.6823 | 0.2437 | 0.6296 | 0.3826 | 0.6783 | 0.2627 | 0.6248 | 0.2715 | 0.6075 | 0.2579 | 0.6404 | 0.3646 | 0.6707 | 0.3207 | 0.6292 | 0.2926 | 0.5841 | 0.3191 | 0.6343 | 0.3784 | 0.6506 | 0.2459 | 0.6490 | 0.3313 | 0.6050 | 0.2195 | 0.6348 | 0.3329 | 0.6364 | 0.3773 | 0.2403 | 0.4597 | 0.3700 | 0.7046 | 0.6493 | 0.7398 | 0.7064 | | 1.3324 | 90.7029 | 40000 | 1.4757 | 0.0013 | 4.3740 | 2.1290 | 0.3304 | 0.6852 | 0.4470 | 0.7163 | 0.2676 | 0.6494 | 0.4321 | 0.7014 | 0.2840 | 0.6297 | 0.2792 | 0.6112 | 0.2784 | 0.6536 | 0.3758 | 0.6853 | 0.3470 | 0.6397 | 0.2836 | 0.5838 | 0.3302 | 0.6511 | 0.3965 | 0.6604 | 0.2511 | 0.6597 | 0.3352 | 0.6558 | 0.2345 | 0.6419 | 0.0863 | 0.3805 | 0.4189 | 0.2606 | 0.4828 | 0.4033 | 0.7202 | 0.6602 | 0.7298 | 0.7267 | | 1.263 | 136.0544 | 60000 | 1.4784 | 0.0013 | 4.3861 | 2.1329 | 0.3207 | 0.6751 | 0.4275 | 0.6999 | 0.2719 | 0.6483 | 0.4223 | 0.6899 | 0.2763 | 0.6338 | 0.2752 | 0.6014 | 0.2785 | 0.6528 | 0.3770 | 0.6798 | 0.3401 | 0.6308 | 0.3102 | 0.5725 | 0.3417 | 0.6461 | 0.3974 | 0.6600 | 0.2657 | 0.6598 | 0.2714 | 0.6686 | 0.2416 | 0.6441 | 0.3684 | 0.6782 | 0.4071 | 0.2588 | 0.4663 | 0.4118 | 0.7277 | 0.6605 | 0.7610 | 0.7240 | | 1.1888 | 181.4059 | 80000 | 1.5073 | 0.0013 | 4.5144 | 2.1745 | 0.3370 | 0.6769 | 0.3840 | 0.6951 | 0.2663 | 0.6444 | 0.4274 | 0.6987 | 0.2671 | 0.6254 | 0.2660 | 0.5993 | 0.2703 | 0.6389 | 0.3555 | 0.6727 | 0.3440 | 0.6310 | 0.3263 | 0.5892 | 0.3298 | 0.6393 | 0.3903 | 0.6569 | 0.2676 | 0.6655 | 0.4326 | 0.6851 | 0.2449 | 0.6462 | 0.3689 | 0.6849 | 0.4143 | 0.2596 | 0.4759 | 0.3976 | 0.7280 | 0.6618 | 0.7646 | 0.7208 | | 1.1401 | 226.7574 | 100000 | 1.5238 | 0.0013 | 4.5894 | 2.1983 | 0.3344 | 0.6828 | 0.4377 | 0.6887 | 0.2711 | 0.6473 | 0.4270 | 0.6977 | 0.2684 | 0.6173 | 0.2735 | 0.5961 | 0.2744 | 0.6365 | 0.3572 | 0.6661 | 0.3377 | 0.6224 | 0.3051 | 0.5802 | 0.3337 | 0.6383 | 0.3790 | 0.6468 | 0.2686 | 0.6594 | 0.3810 | 0.5867 | 0.2505 | 0.6458 | 0.3485 | 0.6866 | 0.4157 | 0.2553 | 0.4973 | 0.3907 | 0.7224 | 0.6580 | 0.7488 | 0.7171 | | 1.1035 | 272.1088 | 120000 | 1.5710 | 0.0013 | 4.8115 | 2.2665 | 0.3319 | 0.6690 | 0.4017 | 0.6756 | 0.2869 | 0.6469 | 0.4084 | 0.6887 | 0.2560 | 0.6048 | 0.2697 | 0.5921 | 0.2766 | 0.6333 | 0.3519 | 0.6525 | 0.3310 | 0.6193 | 0.3101 | 0.5803 | 0.3286 | 0.6391 | 0.3974 | 0.6524 | 0.2679 | 0.6574 | 0.2611 | 0.6827 | 0.2541 | 0.6448 | 0.3777 | 0.6850 | 0.4218 | 0.2600 | 0.4547 | 0.3892 | 0.7235 | 0.6517 | 0.7486 | 0.7108 | | 1.0732 | 317.4603 | 140000 | 1.5963 | 0.0013 | 4.9347 | 2.3030 | 0.3276 | 0.6665 | 0.4038 | 0.6610 | 0.2714 | 0.6390 | 0.3942 | 0.6810 | 0.2579 | 0.6023 | 0.2643 | 0.5904 | 0.2675 | 0.6285 | 0.3478 | 0.6483 | 0.3271 | 0.6237 | 0.3101 | 0.5817 | 0.3249 | 0.6337 | 0.3866 | 0.6459 | 0.2621 | 0.6527 | 0.2926 | 0.6842 | 0.2462 | 0.6416 | 0.3665 | 0.6823 | 0.4058 | 0.2475 | 0.4579 | 0.3821 | 0.7231 | 0.6523 | 0.7451 | 0.7062 | | 1.0485 | 362.8118 | 160000 | 1.6166 | 0.0013 | 5.0360 | 2.3323 | 0.3348 | 0.6715 | 0.3927 | 0.6747 | 0.2878 | 0.6467 | 0.4025 | 0.6911 | 0.2556 | 0.5968 | 0.2590 | 0.5856 | 0.2668 | 0.6309 | 0.3465 | 0.6374 | 0.3375 | 0.6173 | 0.3081 | 0.5802 | 0.3263 | 0.6358 | 0.3717 | 0.6477 | 0.2698 | 0.6594 | 0.2664 | 0.6875 | 0.2537 | 0.6470 | 0.3871 | 0.6892 | 0.4088 | 0.2645 | 0.4867 | 0.3960 | 0.7259 | 0.6569 | 0.7347 | 0.7088 | | 1.0264 | 408.1633 | 180000 | 1.6407 | 0.0013 | 5.1590 | 2.3671 | 0.3316 | 0.6720 | 0.3952 | 0.6720 | 0.2754 | 0.6424 | 0.4033 | 0.6893 | 0.2590 | 0.5971 | 0.2607 | 0.5865 | 0.2590 | 0.6276 | 0.3442 | 0.6562 | 0.3342 | 0.6176 | 0.3020 | 0.5768 | 0.3258 | 0.6388 | 0.3727 | 0.6448 | 0.2613 | 0.6539 | 0.4009 | 0.6823 | 0.2541 | 0.6432 | 0.3713 | 0.6809 | 0.4032 | 0.2569 | 0.4795 | 0.3866 | 0.7269 | 0.6555 | 0.7511 | 0.7086 | | 1.009 | 453.5147 | 200000 | 1.6555 | 0.0013 | 5.2355 | 2.3883 | 0.3332 | 0.6727 | 0.3995 | 0.6660 | 0.2810 | 0.6403 | 0.3995 | 0.6829 | 0.2547 | 0.5907 | 0.2617 | 0.5870 | 0.2604 | 0.6271 | 0.3425 | 0.6148 | 0.3266 | 0.6156 | 0.2731 | 0.5804 | 0.3286 | 0.6367 | 0.3916 | 0.6459 | 0.2640 | 0.6591 | 0.4087 | 0.6786 | 0.2521 | 0.6460 | 0.3788 | 0.6863 | 0.4145 | 0.2634 | 0.4730 | 0.3846 | 0.7263 | 0.6556 | 0.7257 | 0.7086 | ### Framework versions - Transformers 4.44.2 - Pytorch 2.4.0+cu118 - Datasets 2.18.0 - Tokenizers 0.19.1