| [2025-03-23 18:55:03,520][__main__][INFO] - cache_dir: /media/data/tmp | |
| dataset: | |
| name: kamel-usp/aes_enem_dataset | |
| split: JBCS2025 | |
| training_params: | |
| seed: 42 | |
| num_train_epochs: 20 | |
| logging_steps: 100 | |
| metric_for_best_model: QWK | |
| bf16: true | |
| post_training_results: | |
| model_path: /workspace/jbcs2025/outputs/2025-03-23/16-49-40 | |
| experiments: | |
| model: | |
| name: microsoft/phi-4 | |
| type: phi4_classification_lora | |
| num_labels: 6 | |
| output_dir: ./results/phi4-balanced/C3 | |
| logging_dir: ./logs/phi4-balanced/C3 | |
| best_model_dir: ./results/phi4-balanced/C3/best_model | |
| lora_r: 8 | |
| lora_dropout: 0.05 | |
| lora_alpha: 16 | |
| lora_target_modules: all-linear | |
| dataset: | |
| grade_index: 2 | |
| training_id: phi4-balanced-C3 | |
| training_params: | |
| weight_decay: 0.01 | |
| warmup_ratio: 0.1 | |
| learning_rate: 5.0e-05 | |
| train_batch_size: 1 | |
| eval_batch_size: 16 | |
| gradient_accumulation_steps: 16 | |
| gradient_checkpointing: false | |
| [2025-03-23 18:55:03,523][__main__][INFO] - Starting the Fine Tuning training process. | |
| [2025-03-23 18:55:07,176][transformers.tokenization_utils_base][INFO] - loading file vocab.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/vocab.json | |
| [2025-03-23 18:55:07,176][transformers.tokenization_utils_base][INFO] - loading file merges.txt from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/merges.txt | |
| [2025-03-23 18:55:07,176][transformers.tokenization_utils_base][INFO] - loading file tokenizer.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/tokenizer.json | |
| [2025-03-23 18:55:07,176][transformers.tokenization_utils_base][INFO] - loading file added_tokens.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/added_tokens.json | |
| [2025-03-23 18:55:07,176][transformers.tokenization_utils_base][INFO] - loading file special_tokens_map.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/special_tokens_map.json | |
| [2025-03-23 18:55:07,176][transformers.tokenization_utils_base][INFO] - loading file tokenizer_config.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/tokenizer_config.json | |
| [2025-03-23 18:55:07,176][transformers.tokenization_utils_base][INFO] - loading file chat_template.jinja from cache at None | |
| [2025-03-23 18:55:07,365][__main__][INFO] - Tokenizer function parameters- Padding:longest; Truncation: False | |
| [2025-03-23 18:55:08,457][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 18:55:08,457][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "id2label": { | |
| "0": 0, | |
| "1": 40, | |
| "2": 80, | |
| "3": 120, | |
| "4": 160, | |
| "5": 200 | |
| }, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "label2id": { | |
| "0": 0, | |
| "40": 1, | |
| "80": 2, | |
| "120": 3, | |
| "160": 4, | |
| "200": 5 | |
| }, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 18:55:08,482][transformers.modeling_utils][INFO] - loading weights file model.safetensors from cache at /media/data/tmp/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/model.safetensors.index.json | |
| [2025-03-23 18:55:08,482][transformers.modeling_utils][INFO] - Will use torch_dtype=torch.bfloat16 as defined in model's config object | |
| [2025-03-23 18:55:08,482][transformers.modeling_utils][INFO] - Instantiating Phi3ForSequenceClassification model under default dtype torch.bfloat16. | |
| [2025-03-23 18:55:30,833][transformers.modeling_utils][INFO] - Some weights of the model checkpoint at microsoft/phi-4 were not used when initializing Phi3ForSequenceClassification: ['lm_head.weight'] | |
| - This IS expected if you are initializing Phi3ForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model). | |
| - This IS NOT expected if you are initializing Phi3ForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). | |
| [2025-03-23 18:55:30,833][transformers.modeling_utils][WARNING] - Some weights of Phi3ForSequenceClassification were not initialized from the model checkpoint at microsoft/phi-4 and are newly initialized: ['score.weight'] | |
| You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. | |
| [2025-03-23 18:55:32,730][__main__][INFO] - None | |
| [2025-03-23 18:55:32,731][transformers.training_args][INFO] - PyTorch: setting up devices | |
| [2025-03-23 18:55:32,797][__main__][INFO] - Total steps: 620. Number of warmup steps: 62 | |
| [2025-03-23 18:55:32,802][transformers.trainer][INFO] - You have loaded a model on multiple GPUs. `is_model_parallel` attribute will be force-set to `True` to avoid any unexpected behavior such as device placement mismatching. | |
| [2025-03-23 18:55:32,824][transformers.trainer][INFO] - Using auto half precision backend | |
| [2025-03-23 18:55:32,825][transformers.trainer][WARNING] - No label_names provided for model class `PeftModelForSequenceClassification`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead. | |
| [2025-03-23 18:55:32,852][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 18:55:32,864][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 18:55:32,865][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 18:55:32,865][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 18:55:59,524][transformers][INFO] - {'accuracy': 0.30303030303030304, 'RMSE': 59.59458995173263, 'QWK': 0.07447114004020294, 'HDIV': 0.16666666666666663, 'Macro_F1': 0.0989451476793249, 'Micro_F1': 0.30303030303030304, 'Weighted_F1': 0.15414269275028772, 'Macro_F1_(ignoring_nan)': np.float64(0.2968354430379747)} | |
| [2025-03-23 18:55:59,527][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 18:55:59,781][transformers.trainer][INFO] - The following columns in the training set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 18:55:59,813][transformers.trainer][INFO] - ***** Running training ***** | |
| [2025-03-23 18:55:59,813][transformers.trainer][INFO] - Num examples = 500 | |
| [2025-03-23 18:55:59,813][transformers.trainer][INFO] - Num Epochs = 20 | |
| [2025-03-23 18:55:59,813][transformers.trainer][INFO] - Instantaneous batch size per device = 1 | |
| [2025-03-23 18:55:59,813][transformers.trainer][INFO] - Total train batch size (w. parallel, distributed & accumulation) = 16 | |
| [2025-03-23 18:55:59,814][transformers.trainer][INFO] - Gradient Accumulation steps = 16 | |
| [2025-03-23 18:55:59,814][transformers.trainer][INFO] - Total optimization steps = 620 | |
| [2025-03-23 18:55:59,816][transformers.trainer][INFO] - Number of trainable parameters = 27,883,520 | |
| [2025-03-23 19:02:59,972][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 19:02:59,974][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 19:02:59,974][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 19:02:59,974][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 19:03:26,528][transformers][INFO] - {'accuracy': 0.30303030303030304, 'RMSE': 61.987290975039734, 'QWK': 0.0, 'HDIV': 0.18939393939393945, 'Macro_F1': 0.07751937984496124, 'Micro_F1': 0.30303030303030304, 'Weighted_F1': 0.14094432699083861, 'Macro_F1_(ignoring_nan)': np.float64(0.46511627906976744)} | |
| [2025-03-23 19:03:26,529][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 19:03:26,532][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-32 | |
| [2025-03-23 19:03:26,946][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 19:03:26,947][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 19:10:28,123][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 19:10:28,125][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 19:10:28,125][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 19:10:28,125][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 19:10:54,686][transformers][INFO] - {'accuracy': 0.4696969696969697, 'RMSE': 43.90071442781686, 'QWK': 0.23323103901797437, 'HDIV': 0.007575757575757569, 'Macro_F1': 0.17229554865424432, 'Micro_F1': 0.4696969696969697, 'Weighted_F1': 0.37198851872764915, 'Macro_F1_(ignoring_nan)': np.float64(0.5168866459627329)} | |
| [2025-03-23 19:10:54,687][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 19:10:54,689][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-64 | |
| [2025-03-23 19:10:55,513][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 19:10:55,513][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 19:10:56,259][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-32] due to args.save_total_limit | |
| [2025-03-23 19:17:57,083][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 19:17:57,085][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 19:17:57,085][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 19:17:57,085][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 19:18:23,623][transformers][INFO] - {'accuracy': 0.5303030303030303, 'RMSE': 44.312936752559786, 'QWK': 0.30997095837366895, 'HDIV': 0.037878787878787845, 'Macro_F1': 0.2070417145044011, 'Micro_F1': 0.5303030303030303, 'Weighted_F1': 0.4381762516090874, 'Macro_F1_(ignoring_nan)': np.float64(0.6211251435132032)} | |
| [2025-03-23 19:18:23,624][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 19:18:23,626][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-96 | |
| [2025-03-23 19:18:28,949][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 19:18:28,949][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 19:18:29,655][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-64] due to args.save_total_limit | |
| [2025-03-23 19:25:30,121][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 19:25:30,123][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 19:25:30,123][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 19:25:30,123][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 19:25:56,539][transformers][INFO] - {'accuracy': 0.42424242424242425, 'RMSE': 42.211588240886904, 'QWK': 0.48123195380173234, 'HDIV': 0.030303030303030276, 'Macro_F1': 0.2731339395661646, 'Micro_F1': 0.42424242424242425, 'Weighted_F1': 0.3931376854650512, 'Macro_F1_(ignoring_nan)': np.float64(0.4097009093492469)} | |
| [2025-03-23 19:25:56,540][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 19:25:56,543][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-128 | |
| [2025-03-23 19:25:56,826][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 19:25:56,827][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 19:25:57,638][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-96] due to args.save_total_limit | |
| [2025-03-23 19:32:57,872][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 19:32:57,874][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 19:32:57,874][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 19:32:57,874][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 19:33:24,216][transformers][INFO] - {'accuracy': 0.4621212121212121, 'RMSE': 50.81159495448044, 'QWK': 0.3757548845470693, 'HDIV': 0.06060606060606055, 'Macro_F1': 0.22010546642899587, 'Micro_F1': 0.4621212121212121, 'Weighted_F1': 0.42268946145951497, 'Macro_F1_(ignoring_nan)': np.float64(0.44021093285799173)} | |
| [2025-03-23 19:33:24,217][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 19:33:24,219][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-160 | |
| [2025-03-23 19:33:24,513][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 19:33:24,514][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 19:40:25,539][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 19:40:25,541][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 19:40:25,541][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 19:40:25,541][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 19:40:51,871][transformers][INFO] - {'accuracy': 0.48484848484848486, 'RMSE': 49.848254581765296, 'QWK': 0.2355068369307266, 'HDIV': 0.08333333333333337, 'Macro_F1': 0.252997799779978, 'Micro_F1': 0.48484848484848486, 'Weighted_F1': 0.4096209620962097, 'Macro_F1_(ignoring_nan)': np.float64(0.505995599559956)} | |
| [2025-03-23 19:40:51,871][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 19:40:51,875][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-192 | |
| [2025-03-23 19:40:52,193][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 19:40:52,194][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 19:40:52,939][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-160] due to args.save_total_limit | |
| [2025-03-23 19:47:53,368][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 19:47:53,370][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 19:47:53,370][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 19:47:53,370][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 19:48:19,749][transformers][INFO] - {'accuracy': 0.44696969696969696, 'RMSE': 46.31905164675272, 'QWK': 0.3573550445593575, 'HDIV': 0.05303030303030298, 'Macro_F1': 0.26573426573426573, 'Micro_F1': 0.44696969696969696, 'Weighted_F1': 0.40464081373172284, 'Macro_F1_(ignoring_nan)': np.float64(0.31888111888111886)} | |
| [2025-03-23 19:48:19,749][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 19:48:19,753][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-224 | |
| [2025-03-23 19:48:20,074][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 19:48:20,075][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 19:48:20,877][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-192] due to args.save_total_limit | |
| [2025-03-23 19:55:21,142][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 19:55:21,144][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 19:55:21,144][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 19:55:21,144][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 19:55:47,815][transformers][INFO] - {'accuracy': 0.4090909090909091, 'RMSE': 53.25752187591567, 'QWK': 0.38462764473841493, 'HDIV': 0.06818181818181823, 'Macro_F1': 0.2607843137254902, 'Micro_F1': 0.4090909090909091, 'Weighted_F1': 0.3992869875222817, 'Macro_F1_(ignoring_nan)': np.float64(0.39117647058823535)} | |
| [2025-03-23 19:55:47,815][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 19:55:47,818][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-256 | |
| [2025-03-23 19:55:48,156][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 19:55:48,157][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 19:55:49,761][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-224] due to args.save_total_limit | |
| [2025-03-23 20:02:50,295][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 20:02:50,298][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 20:02:50,298][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 20:02:50,298][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 20:03:16,795][transformers][INFO] - {'accuracy': 0.4772727272727273, 'RMSE': 48.865926655275736, 'QWK': 0.2592297174111212, 'HDIV': 0.0757575757575758, 'Macro_F1': 0.2523213789268835, 'Micro_F1': 0.4772727272727273, 'Weighted_F1': 0.4081406222357015, 'Macro_F1_(ignoring_nan)': np.float64(0.3784820683903253)} | |
| [2025-03-23 20:03:16,796][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 20:03:16,799][transformers.trainer][INFO] - Saving model checkpoint to /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-288 | |
| [2025-03-23 20:03:17,084][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 20:03:17,085][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 20:03:18,370][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-256] due to args.save_total_limit | |
| [2025-03-23 20:03:18,416][transformers.trainer][INFO] - | |
| Training completed. Do not forget to share your model on huggingface.co/models =) | |
| [2025-03-23 20:03:18,416][transformers.trainer][INFO] - Loading best model from /workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-128 (score: 0.48123195380173234). | |
| [2025-03-23 20:03:18,814][transformers.trainer][INFO] - Deleting older checkpoint [/workspace/jbcs2025/outputs/2025-03-23/18-55-03/results/phi4-balanced/C3/checkpoint-288] due to args.save_total_limit | |
| [2025-03-23 20:03:18,866][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 20:03:18,868][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 20:03:18,868][transformers.trainer][INFO] - Num examples = 132 | |
| [2025-03-23 20:03:18,868][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 20:03:45,321][transformers][INFO] - {'accuracy': 0.42424242424242425, 'RMSE': 42.211588240886904, 'QWK': 0.48123195380173234, 'HDIV': 0.030303030303030276, 'Macro_F1': 0.2731339395661646, 'Micro_F1': 0.42424242424242425, 'Weighted_F1': 0.3931376854650512, 'Macro_F1_(ignoring_nan)': np.float64(0.4097009093492469)} | |
| [2025-03-23 20:03:45,324][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 20:03:45,325][__main__][INFO] - Training completed successfully. | |
| [2025-03-23 20:03:45,325][__main__][INFO] - Running on Test | |
| [2025-03-23 20:03:45,326][transformers.trainer][INFO] - The following columns in the evaluation set don't have a corresponding argument in `PeftModelForSequenceClassification.forward` and have been ignored: essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text. If essay_text, id, prompt, grades, id_prompt, essay_year, reference, supporting_text are not expected by `PeftModelForSequenceClassification.forward`, you can safely ignore this message. | |
| [2025-03-23 20:03:45,328][transformers.trainer][INFO] - | |
| ***** Running Evaluation ***** | |
| [2025-03-23 20:03:45,328][transformers.trainer][INFO] - Num examples = 138 | |
| [2025-03-23 20:03:45,328][transformers.trainer][INFO] - Batch size = 16 | |
| [2025-03-23 20:04:13,300][transformers][INFO] - {'accuracy': 0.3333333333333333, 'RMSE': 49.22589162013322, 'QWK': 0.29381519024533553, 'HDIV': 0.036231884057971064, 'Macro_F1': 0.20859754797807895, 'Micro_F1': 0.3333333333333333, 'Weighted_F1': 0.2747500757119841, 'Macro_F1_(ignoring_nan)': np.float64(0.3128963219671184)} | |
| [2025-03-23 20:04:13,301][tensorboardX.summary][INFO] - Summary name eval/Macro_F1_(ignoring_nan) is illegal; using eval/Macro_F1__ignoring_nan_ instead. | |
| [2025-03-23 20:04:13,303][transformers.trainer][INFO] - Saving model checkpoint to ./results/phi4-balanced/C3/best_model | |
| [2025-03-23 20:04:13,633][transformers.configuration_utils][INFO] - loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--microsoft--phi-4/snapshots/187ef0342fff0eb3333be9f00389385e95ef0b61/config.json | |
| [2025-03-23 20:04:13,633][transformers.configuration_utils][INFO] - Model config Phi3Config { | |
| "architectures": [ | |
| "Phi3ForCausalLM" | |
| ], | |
| "attention_bias": false, | |
| "attention_dropout": 0.0, | |
| "bos_token_id": 100257, | |
| "embd_pdrop": 0.0, | |
| "eos_token_id": 100265, | |
| "hidden_act": "silu", | |
| "hidden_size": 5120, | |
| "initializer_range": 0.02, | |
| "intermediate_size": 17920, | |
| "max_position_embeddings": 16384, | |
| "model_type": "phi3", | |
| "num_attention_heads": 40, | |
| "num_hidden_layers": 40, | |
| "num_key_value_heads": 10, | |
| "original_max_position_embeddings": 16384, | |
| "pad_token_id": 100349, | |
| "partial_rotary_factor": 1.0, | |
| "resid_pdrop": 0.0, | |
| "rms_norm_eps": 1e-05, | |
| "rope_scaling": null, | |
| "rope_theta": 250000, | |
| "sliding_window": null, | |
| "tie_word_embeddings": false, | |
| "torch_dtype": "bfloat16", | |
| "transformers_version": "4.50.0", | |
| "use_cache": true, | |
| "vocab_size": 100352 | |
| } | |
| [2025-03-23 20:04:14,250][__main__][INFO] - Fine Tuning Finished. | |