Hugging Face's logo

EricWesthoff
/
phi-1_5-finetuned-SQL

Text Generation
Transformers
PyTorch
mixformer-sequential
Generated from Trainer
custom_code
Model card Files Community
EricWesthoff commited on
Commit
4c3b9f1
·
1 Parent(s): 8e99796

End of training

Browse files
Files changed (1) hide show
  1. README.md +125 -65
README.md CHANGED
@@ -15,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
15
 
16
  This model is a fine-tuned version of [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 1.5295
19
 
20
  ## Model description
21
 
@@ -34,78 +34,138 @@ More information needed
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
37
- - learning_rate: 0.0002
38
  - train_batch_size: 4
39
  - eval_batch_size: 4
40
  - seed: 42
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
- - training_steps: 6000
44
 
45
  ### Training results
46
 
47
- | Training Loss | Epoch | Step | Validation Loss |
48
- |:-------------:|:-----:|:----:|:---------------:|
49
- | 2.3757 | 0.04 | 100 | 2.0747 |
50
- | 2.0269 | 0.08 | 200 | 1.9990 |
51
- | 1.9535 | 0.12 | 300 | 1.9450 |
52
- | 1.9136 | 0.16 | 400 | 1.9067 |
53
- | 1.892 | 0.2 | 500 | 1.8757 |
54
- | 1.8753 | 0.24 | 600 | 1.8574 |
55
- | 1.8507 | 0.28 | 700 | 1.8359 |
56
- | 1.8759 | 0.32 | 800 | 1.8167 |
57
- | 1.8166 | 0.36 | 900 | 1.8054 |
58
- | 1.8224 | 0.4 | 1000 | 1.7818 |
59
- | 1.7852 | 0.44 | 1100 | 1.7814 |
60
- | 1.8164 | 0.48 | 1200 | 1.7664 |
61
- | 1.7632 | 0.52 | 1300 | 1.7598 |
62
- | 1.8485 | 0.56 | 1400 | 1.7439 |
63
- | 1.7712 | 0.6 | 1500 | 1.7303 |
64
- | 1.7632 | 0.64 | 1600 | 1.7277 |
65
- | 1.7378 | 0.68 | 1700 | 1.7135 |
66
- | 1.7581 | 0.72 | 1800 | 1.7075 |
67
- | 1.7261 | 0.76 | 1900 | 1.6933 |
68
- | 1.7243 | 0.8 | 2000 | 1.6891 |
69
- | 1.7311 | 0.84 | 2100 | 1.6837 |
70
- | 1.7554 | 0.88 | 2200 | 1.6808 |
71
- | 1.7026 | 0.92 | 2300 | 1.6646 |
72
- | 1.7193 | 0.96 | 2400 | 1.6664 |
73
- | 1.6861 | 1.0 | 2500 | 1.6577 |
74
- | 1.68 | 1.04 | 2600 | 1.6470 |
75
- | 1.5931 | 1.08 | 2700 | 1.6425 |
76
- | 1.6655 | 1.12 | 2800 | 1.6352 |
77
- | 1.629 | 1.16 | 2900 | 1.6298 |
78
- | 1.6567 | 1.2 | 3000 | 1.6236 |
79
- | 1.6225 | 1.24 | 3100 | 1.6242 |
80
- | 1.6249 | 1.28 | 3200 | 1.6150 |
81
- | 1.6263 | 1.32 | 3300 | 1.6077 |
82
- | 1.6055 | 1.36 | 3400 | 1.6034 |
83
- | 1.6338 | 1.4 | 3500 | 1.5996 |
84
- | 1.6032 | 1.44 | 3600 | 1.5947 |
85
- | 1.6447 | 1.48 | 3700 | 1.5882 |
86
- | 1.6063 | 1.52 | 3800 | 1.5877 |
87
- | 1.5933 | 1.56 | 3900 | 1.5850 |
88
- | 1.6267 | 1.6 | 4000 | 1.5814 |
89
- | 1.6151 | 1.64 | 4100 | 1.5709 |
90
- | 1.6047 | 1.68 | 4200 | 1.5683 |
91
- | 1.5811 | 1.72 | 4300 | 1.5661 |
92
- | 1.5877 | 1.76 | 4400 | 1.5648 |
93
- | 1.6321 | 1.8 | 4500 | 1.5645 |
94
- | 1.5969 | 1.84 | 4600 | 1.5584 |
95
- | 1.5971 | 1.88 | 4700 | 1.5565 |
96
- | 1.622 | 1.92 | 4800 | 1.5547 |
97
- | 1.6265 | 1.96 | 4900 | 1.5496 |
98
- | 1.6145 | 2.0 | 5000 | 1.5466 |
99
- | 1.526 | 2.04 | 5100 | 1.5427 |
100
- | 1.5793 | 2.08 | 5200 | 1.5390 |
101
- | 1.5714 | 2.12 | 5300 | 1.5375 |
102
- | 1.5228 | 2.16 | 5400 | 1.5360 |
103
- | 1.5383 | 2.2 | 5500 | 1.5343 |
104
- | 1.5117 | 2.24 | 5600 | 1.5322 |
105
- | 1.5427 | 2.28 | 5700 | 1.5316 |
106
- | 1.4959 | 2.32 | 5800 | 1.5306 |
107
- | 1.5456 | 2.36 | 5900 | 1.5299 |
108
- | 1.5175 | 2.4 | 6000 | 1.5295 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
109
 
110
 
111
  ### Framework versions
 
15
 
16
  This model is a fine-tuned version of [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 1.4066
19
 
20
  ## Model description
21
 
 
34
  ### Training hyperparameters
35
 
36
  The following hyperparameters were used during training:
37
+ - learning_rate: 0.0006
38
  - train_batch_size: 4
39
  - eval_batch_size: 4
40
  - seed: 42
41
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
42
  - lr_scheduler_type: linear
43
+ - training_steps: 12000
44
 
45
  ### Training results
46
 
47
+ | Training Loss | Epoch | Step | Validation Loss |
48
+ |:-------------:|:-----:|:-----:|:---------------:|
49
+ | 2.3757 | 0.04 | 100 | 2.0747 |
50
+ | 2.0269 | 0.08 | 200 | 1.9990 |
51
+ | 1.9535 | 0.12 | 300 | 1.9450 |
52
+ | 1.9136 | 0.16 | 400 | 1.9067 |
53
+ | 1.892 | 0.2 | 500 | 1.8757 |
54
+ | 1.8753 | 0.24 | 600 | 1.8574 |
55
+ | 1.8507 | 0.28 | 700 | 1.8359 |
56
+ | 1.8759 | 0.32 | 800 | 1.8167 |
57
+ | 1.8166 | 0.36 | 900 | 1.8054 |
58
+ | 1.8224 | 0.4 | 1000 | 1.7818 |
59
+ | 1.7852 | 0.44 | 1100 | 1.7814 |
60
+ | 1.8164 | 0.48 | 1200 | 1.7664 |
61
+ | 1.7632 | 0.52 | 1300 | 1.7598 |
62
+ | 1.8485 | 0.56 | 1400 | 1.7439 |
63
+ | 1.7712 | 0.6 | 1500 | 1.7303 |
64
+ | 1.7632 | 0.64 | 1600 | 1.7277 |
65
+ | 1.7378 | 0.68 | 1700 | 1.7135 |
66
+ | 1.7581 | 0.72 | 1800 | 1.7075 |
67
+ | 1.7261 | 0.76 | 1900 | 1.6933 |
68
+ | 1.7243 | 0.8 | 2000 | 1.6891 |
69
+ | 1.7311 | 0.84 | 2100 | 1.6837 |
70
+ | 1.7554 | 0.88 | 2200 | 1.6808 |
71
+ | 1.7026 | 0.92 | 2300 | 1.6646 |
72
+ | 1.7193 | 0.96 | 2400 | 1.6664 |
73
+ | 1.6861 | 1.0 | 2500 | 1.6577 |
74
+ | 1.68 | 1.04 | 2600 | 1.6470 |
75
+ | 1.5931 | 1.08 | 2700 | 1.6425 |
76
+ | 1.6655 | 1.12 | 2800 | 1.6352 |
77
+ | 1.629 | 1.16 | 2900 | 1.6298 |
78
+ | 1.6567 | 1.2 | 3000 | 1.6236 |
79
+ | 1.6225 | 1.24 | 3100 | 1.6242 |
80
+ | 1.6249 | 1.28 | 3200 | 1.6150 |
81
+ | 1.6263 | 1.32 | 3300 | 1.6077 |
82
+ | 1.6055 | 1.36 | 3400 | 1.6034 |
83
+ | 1.6338 | 1.4 | 3500 | 1.5996 |
84
+ | 1.6032 | 1.44 | 3600 | 1.5947 |
85
+ | 1.6447 | 1.48 | 3700 | 1.5882 |
86
+ | 1.6063 | 1.52 | 3800 | 1.5877 |
87
+ | 1.5933 | 1.56 | 3900 | 1.5850 |
88
+ | 1.6267 | 1.6 | 4000 | 1.5814 |
89
+ | 1.6151 | 1.64 | 4100 | 1.5709 |
90
+ | 1.6047 | 1.68 | 4200 | 1.5683 |
91
+ | 1.5811 | 1.72 | 4300 | 1.5661 |
92
+ | 1.5877 | 1.76 | 4400 | 1.5648 |
93
+ | 1.6321 | 1.8 | 4500 | 1.5645 |
94
+ | 1.5969 | 1.84 | 4600 | 1.5584 |
95
+ | 1.5971 | 1.88 | 4700 | 1.5565 |
96
+ | 1.622 | 1.92 | 4800 | 1.5547 |
97
+ | 1.6265 | 1.96 | 4900 | 1.5496 |
98
+ | 1.6145 | 2.0 | 5000 | 1.5466 |
99
+ | 1.526 | 2.04 | 5100 | 1.5427 |
100
+ | 1.5793 | 2.08 | 5200 | 1.5390 |
101
+ | 1.5714 | 2.12 | 5300 | 1.5375 |
102
+ | 1.5228 | 2.16 | 5400 | 1.5360 |
103
+ | 1.5383 | 2.2 | 5500 | 1.5343 |
104
+ | 1.5117 | 2.24 | 5600 | 1.5322 |
105
+ | 1.5427 | 2.28 | 5700 | 1.5316 |
106
+ | 1.4959 | 2.32 | 5800 | 1.5306 |
107
+ | 1.5456 | 2.36 | 5900 | 1.5299 |
108
+ | 1.5175 | 2.4 | 6000 | 1.5295 |
109
+ | 1.5823 | 2.44 | 6100 | 1.5498 |
110
+ | 1.5615 | 2.48 | 6200 | 1.5447 |
111
+ | 1.5326 | 2.52 | 6300 | 1.5463 |
112
+ | 1.567 | 2.56 | 6400 | 1.5450 |
113
+ | 1.5243 | 2.6 | 6500 | 1.5456 |
114
+ | 1.5214 | 2.64 | 6600 | 1.5383 |
115
+ | 1.6086 | 2.68 | 6700 | 1.5393 |
116
+ | 1.5391 | 2.72 | 6800 | 1.5285 |
117
+ | 1.5224 | 2.76 | 6900 | 1.5318 |
118
+ | 1.5567 | 2.8 | 7000 | 1.5292 |
119
+ | 1.5525 | 2.84 | 7100 | 1.5207 |
120
+ | 1.5399 | 2.88 | 7200 | 1.5135 |
121
+ | 1.5399 | 2.92 | 7300 | 1.5104 |
122
+ | 1.5765 | 2.96 | 7400 | 1.5085 |
123
+ | 1.556 | 3.0 | 7500 | 1.5042 |
124
+ | 1.4977 | 3.04 | 7600 | 1.4997 |
125
+ | 1.4818 | 3.08 | 7700 | 1.4930 |
126
+ | 1.4912 | 3.12 | 7800 | 1.4908 |
127
+ | 1.517 | 3.16 | 7900 | 1.4933 |
128
+ | 1.4971 | 3.2 | 8000 | 1.4857 |
129
+ | 1.4827 | 3.24 | 8100 | 1.4805 |
130
+ | 1.5096 | 3.28 | 8200 | 1.4804 |
131
+ | 1.4788 | 3.32 | 8300 | 1.4756 |
132
+ | 1.457 | 3.36 | 8400 | 1.4728 |
133
+ | 1.4819 | 3.4 | 8500 | 1.4717 |
134
+ | 1.5241 | 3.44 | 8600 | 1.4678 |
135
+ | 1.5081 | 3.48 | 8700 | 1.4676 |
136
+ | 1.5173 | 3.52 | 8800 | 1.4657 |
137
+ | 1.4765 | 3.56 | 8900 | 1.4643 |
138
+ | 1.4691 | 3.6 | 9000 | 1.4603 |
139
+ | 1.5034 | 3.64 | 9100 | 1.4577 |
140
+ | 1.4997 | 3.68 | 9200 | 1.4552 |
141
+ | 1.4849 | 3.72 | 9300 | 1.4504 |
142
+ | 1.5144 | 3.76 | 9400 | 1.4518 |
143
+ | 1.4972 | 3.8 | 9500 | 1.4469 |
144
+ | 1.4695 | 3.84 | 9600 | 1.4474 |
145
+ | 1.5088 | 3.88 | 9700 | 1.4468 |
146
+ | 1.4772 | 3.92 | 9800 | 1.4418 |
147
+ | 1.5207 | 3.96 | 9900 | 1.4390 |
148
+ | 1.5088 | 4.0 | 10000 | 1.4378 |
149
+ | 1.4915 | 4.04 | 10100 | 1.4324 |
150
+ | 1.4356 | 4.08 | 10200 | 1.4305 |
151
+ | 1.4388 | 4.12 | 10300 | 1.4268 |
152
+ | 1.4004 | 4.16 | 10400 | 1.4251 |
153
+ | 1.3909 | 4.2 | 10500 | 1.4225 |
154
+ | 1.4284 | 4.24 | 10600 | 1.4218 |
155
+ | 1.4422 | 4.28 | 10700 | 1.4213 |
156
+ | 1.4301 | 4.32 | 10800 | 1.4198 |
157
+ | 1.4309 | 4.36 | 10900 | 1.4174 |
158
+ | 1.415 | 4.4 | 11000 | 1.4147 |
159
+ | 1.4697 | 4.44 | 11100 | 1.4136 |
160
+ | 1.4241 | 4.48 | 11200 | 1.4123 |
161
+ | 1.4416 | 4.52 | 11300 | 1.4100 |
162
+ | 1.4229 | 4.56 | 11400 | 1.4094 |
163
+ | 1.4498 | 4.6 | 11500 | 1.4091 |
164
+ | 1.4023 | 4.64 | 11600 | 1.4083 |
165
+ | 1.4197 | 4.68 | 11700 | 1.4075 |
166
+ | 1.4165 | 4.72 | 11800 | 1.4070 |
167
+ | 1.4103 | 4.76 | 11900 | 1.4067 |
168
+ | 1.4214 | 4.8 | 12000 | 1.4066 |
169
 
170
 
171
  ### Framework versions