noniewiem commited on
Commit
5497460
1 Parent(s): 91666af

End of training

Browse files
all_results.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 33.33,
3
+ "eval_cola_loss": 0.6395143270492554,
4
+ "eval_cola_matthews_correlation": -0.03589254563226399,
5
+ "eval_cola_runtime": 7.7785,
6
+ "eval_cola_samples_per_second": 134.088,
7
+ "eval_cola_steps_per_second": 16.841,
8
+ "eval_samples": 1043,
9
+ "train_loss": 0.6206500174782493,
10
+ "train_runtime": 3140.0413,
11
+ "train_samples": 8551,
12
+ "train_samples_per_second": 1222.914,
13
+ "train_steps_per_second": 4.777
14
+ }
cola_outputs.csv ADDED
@@ -0,0 +1,1044 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ sentence label prediction length
2
+ The sailors rode the breeze clear of the rocks. 1 1 47
3
+ The weights made the rope stretch over the pulley. 1 1 50
4
+ The mechanical doll wriggled itself loose. 1 1 42
5
+ If you had eaten more, you would want less. 1 1 43
6
+ As you eat the most, you want the least. 0 1 40
7
+ The more you would want, the less you would eat. 0 1 48
8
+ I demand that the more John eat, the more he pays. 0 1 50
9
+ Mary listens to the Grateful Dead, she gets depressed. 1 1 54
10
+ The angrier Mary got, the more she looked at pictures. 1 1 54
11
+ The higher the stakes, the lower his expectations are. 1 1 54
12
+ The more Fred is obnoxious, the less attention you should pay to him. 1 1 69
13
+ John was lots more obnoxious than Fred. 1 1 39
14
+ The more people you give beer to, the more people get sick. 1 1 59
15
+ The more does Bill smoke, the more Susan hates him. 0 1 51
16
+ The more pictures of him that appear in the news, the more embarrassed John becomes. 1 1 84
17
+ Every senator seems to become more corrupt, as he talks to more lobbyists. 1 1 74
18
+ Who does John visit Sally because he likes? 0 1 43
19
+ Marianne did not leave. 1 1 23
20
+ He could not] have been working. 1 1 32
21
+ He can not have been working. 1 1 29
22
+ You will believe Bob. 1 1 21
23
+ John has not kissed Mary. 1 1 25
24
+ I said that never in my life had I seen a place like Bangor. 1 1 60
25
+ Mickey looked up it. 0 1 20
26
+ There tended to be a lot of discussion. 1 1 39
27
+ John tried to be a good boy. 1 1 28
28
+ John is eager. 1 1 14
29
+ We want John to win. 1 1 20
30
+ The box contained the ball from the tree. 0 1 41
31
+ The tube was escaped by gas. 0 1 28
32
+ Water bubbled up out of the kettle. 1 1 35
33
+ The tub leaked water. 1 1 21
34
+ What the water did to the bottle was fill it. 0 1 45
35
+ What the water did to the whole bottle was fill it. 0 1 51
36
+ The tank leaked the fluid free. 1 1 31
37
+ John lay the ball in the box. 1 1 29
38
+ John owns the book. 1 1 19
39
+ We persuaded Mary to leave and Sue to stay. 1 1 43
40
+ Most people probably consider, even though the courts didn't actually find, Klaus guilty of murder. 1 1 99
41
+ Mary beautifully plays the violin. 0 1 34
42
+ Clearly, John probably will immediately learn French perfectly. 1 1 63
43
+ Sue gave to Bill a book. 0 1 24
44
+ The men will all leave. 1 1 23
45
+ John went home. 1 1 15
46
+ They represented seriously to the dean Mary as a genuine linguist. 0 1 66
47
+ Us love they. 0 1 13
48
+ It is nice to go abroad. 1 1 24
49
+ Mary intended John to go abroad. 0 1 32
50
+ I remembered having kissed Mary. 1 1 32
51
+ I can't believe Fred won't, either. 1 1 35
52
+ John wants to read Fred's story, and I also want to. 1 1 52
53
+ We wanted to invite someone, but we couldn't decide who to. 0 1 59
54
+ Mary will read Fred's story, and Joe will read Holly's. 1 1 55
55
+ Mary claimed that eating cabbage, Holly shouldn't. 1 1 50
56
+ Mary came to be introduced by the bartender and I also came to be. 0 1 66
57
+ If I can, I will work on it. 1 1 28
58
+ Joe's neuroses bother his patrons, and Sally does too. 1 1 54
59
+ I know which book José didn't read for class, and which book Lilly did it for him. 0 1 82
60
+ This is the book which Bob reviewed, and this is the one which Fred won't do it. 0 1 80
61
+ I know which book Mag read, and which book Bob said that you hadn't. 1 1 68
62
+ I know which book Mag read, and which book Bob read my report that you hadn't. 0 1 78
63
+ I'm sure I would like him to eat fruit more than I would cookies. 1 1 65
64
+ Rusty talked about himself only after Mary did talk about him. 1 1 62
65
+ Fred talked about everything before Rusty did talk about something. 1 1 67
66
+ John often meets Mary. 1 1 22
67
+ The problem perceives easily. 0 1 29
68
+ A hundred men surrounded the fort. 1 1 34
69
+ We elected me. 1 1 14
70
+ Which report that John was incompetent did he submit? 0 1 53
71
+ Mary has always preferred lemons to limes. 1 1 42
72
+ He let the cats which were whining out. 1 1 39
73
+ What did Bill buy? 1 1 18
74
+ Mary saw the boy walking toward the railroad station. 1 1 53
75
+ A proof that the claim had been. made was giver that John had lied. 0 1 67
76
+ He attributed to a short circuit which was caused by an overloaded transducer the fire which destroyed most of my factory. 1 1 122
77
+ The mayor regarded as being absurd the proposal to build a sidewalk from Dartmouth to Smith. 0 1 92
78
+ I want that Bill left to remain a secret. 0 1 41
79
+ I know a man who Tom drives as drives. 0 1 38
80
+ Drowning cats, which is against the law, are hard to rescue. 0 1 60
81
+ Muriel said nothing else than that she had been insulted. 1 1 57
82
+ Himself is understood by Rutherford. 0 1 36
83
+ I feel that Arch will show up. 1 1 30
84
+ The proof this set is recursive is difficult. 0 1 45
85
+ The madrigals which Henry plays the lute and sings sound lousy. 0 1 63
86
+ Tom picked these grapes, and I washed some turnips, and Suzie will prepare these grapes. 1 1 88
87
+ Where did you go and who ate what? 1 1 34
88
+ Which boy's did we elect guardian's employer president? 0 1 55
89
+ How sane is Peter? 1 1 18
90
+ I live at the place where Route 150 crosses the River and my dad lives at the place where Route 150 crosses the Hudson River too. 1 1 129
91
+ I live at the place where Route 150 crosses the Hudson River and my dad lives at it too. 0 1 88
92
+ Who is she trying to make up to now? 1 1 36
93
+ Wind was gotten of a plot to negotiate an honorable end to the war in Vietnam. 0 1 78
94
+ Mike talked about politics yesterday to my friends. 1 1 51
95
+ It was expected by the reporters that the principal would fire some teacher. 1 1 76
96
+ Which hat did Mike quip that she never wore? 0 1 44
97
+ Which girl did Mike quip never wore this hat? 0 1 45
98
+ We donated wire for the convicts to build cages with. 1 1 53
99
+ I won't have some money. 0 1 24
100
+ Do you believe the claim that somebody was looking for something? 1 1 65
101
+ I won't ask you to believe that he tried to force me to give her any money. 1 1 75
102
+ That Sam sometimes didn't sleep must have pleased somebody. 1 1 59
103
+ I talked to Winston about himself. 1 1 34
104
+ That the fuzz wanted him worried John, but that the fuzz wanted her didn't worry Mary. 1 1 86
105
+ I'll work on it if Sam will be working on it. 1 1 45
106
+ I'll work on it if I can. 1 1 25
107
+ Here's a knife with which for you to cut up the onions. 0 1 55
108
+ Fluffy is sick, which not everybody knows. 1 1 42
109
+ Maxwell is quite a doctor. 1 1 26
110
+ The younger woman might have been tall and, and the older one definitely was, blond. 0 1 84
111
+ Sally is tall, and may be blond, and Sheila is short, and definitely is, blond. 1 1 79
112
+ I have to try to finish grading some papers. 1 1 44
113
+ The socks are ready for for you to put on to be planned. 0 1 56
114
+ It is easy to play sonatas on this violin. 1 1 42
115
+ This violin is difficult to play sonatas on. 1 1 44
116
+ My mother is easy to please my father and. 0 1 42
117
+ Poor Bill, it had started to rain and he had no umbrella. 1 1 57
118
+ That the cops spoke to the janitor about it yesterday is terrible, that robbery. 0 1 80
119
+ Every student, and he wears socks, is a swinger. 0 1 48
120
+ That girl was given my binoculars by him. 1 1 41
121
+ Bill didn't allege that Roger had eaten anything. 1 1 49
122
+ Nobody who hates to eat anything should work in a delicatessen. 1 1 63
123
+ Everybody around here who ever buys anything on credit talks in his sleep. 1 1 74
124
+ I can't remember the name of somebody who had misgivings. 0 1 57
125
+ No writer, and no playwright, meets in Vienna. 0 1 46
126
+ No writer, nor any playwright, meets in Vienna. 0 1 47
127
+ That you will marry any student is not certain. 1 1 47
128
+ Felicia kicked the ball off the bench. 1 1 38
129
+ I sent the package halfway around the world. 1 1 44
130
+ Sam gave the ball out of the basket. 0 1 36
131
+ Sam offered the ball out of the basket. 0 1 39
132
+ Park Square has a festive air. 1 1 30
133
+ The worker will have a job. 1 1 27
134
+ No one can forgive that comment to you. 0 1 39
135
+ We launched the rocket to the moon, but it blew up before it got there. 1 1 71
136
+ Sarah promised Catherine her old car, but then gave it to her son instead. 1 1 74
137
+ I lent the book partway to Tony. 0 1 32
138
+ The farmer loaded the cart with apples. 1 1 39
139
+ The farmer dumped the cart with apples. 0 1 39
140
+ Martha carved the baby a toy out of wood. 1 1 41
141
+ The bread cuts easily. 1 1 22
142
+ Janet broke Bill on the finger. 0 1 31
143
+ Janet broke the cup. 1 1 20
144
+ The visitor rang the bell. 1 1 26
145
+ We pulled free. 1 1 15
146
+ That movie always shocks people. 1 1 32
147
+ That movie always shocks. 1 1 25
148
+ Sharon came the room. 0 1 21
149
+ Bill sent a package to Tom. 1 1 27
150
+ That acorn will grow into an oak tree. 1 1 38
151
+ He turned into a frog. 1 1 22
152
+ I mixed the sugar into the butter. 1 1 34
153
+ Brian threw the fence with the stick. 0 1 37
154
+ Mira condemned Terry for the accident. 1 1 38
155
+ We investigated the area for bombs. 1 1 35
156
+ I sensed his eagerness. 1 1 23
157
+ They praised the dedication in the volunteers. 0 1 46
158
+ The earth was believed to be round. 1 1 35
159
+ Sarah smiled a charming smile. 1 1 30
160
+ Sandra beamed a cheerful welcome. 1 1 33
161
+ You've really lived it up. 1 1 26
162
+ Paperback books lift onto the table easily. 0 1 43
163
+ The books lifted onto the table. 0 1 32
164
+ Jessica loaded boxes under the wagon. 1 1 37
165
+ Jessica loaded boxes on the wagon. 1 1 34
166
+ Jessica crammed boxes at the truck. 0 1 35
167
+ Lora buttered at the toast. 0 1 27
168
+ Carla shoveled the walk. 1 1 24
169
+ Nora sent the book. 1 1 19
170
+ Nora sent Peter the book. 1 1 25
171
+ Carla slid the book. 1 1 20
172
+ Carla slid at the book. 0 1 23
173
+ Amanda carried the package to Pamela. 1 1 37
174
+ Packages drive easily to New York. 0 1 34
175
+ The chair pushed. 0 1 17
176
+ We offered a job to her. 1 1 24
177
+ A job offered. 0 1 14
178
+ Brown presented a plaque to Jones. 1 1 34
179
+ Carmen bought Mary a dress. 1 1 27
180
+ Carmen obtained the spare part at the hardware store. 1 1 53
181
+ Michelle kept the desk with the papers. 0 1 39
182
+ Frances hid the presents in the drawer. 1 1 39
183
+ The needle poked the cloth. 1 1 27
184
+ Carrie touched that cat. 1 1 24
185
+ Herman whipped the sugar and the cream. 0 1 39
186
+ Linda taped the picture to the wall. 1 1 36
187
+ Linda taped the picture onto the wall. 1 1 38
188
+ The child and her mother clung together. 1 1 40
189
+ This flyer and that flyer differ. 1 1 33
190
+ This flyer and that flyer differ apart. 0 1 39
191
+ The jeweller scribbled the contract with his name. 0 1 50
192
+ The gardener grew that acorn into an oak tree. 1 1 46
193
+ I shaped a loaf. 0 1 16
194
+ The children amused. 0 1 20
195
+ Susan whispered the news. 1 1 25
196
+ Susan whispered at Rachel. 1 1 26
197
+ Ellen said that melons were selling well. 1 1 41
198
+ Ellen said about the present conditions. 0 1 40
199
+ Ellen warned Helen against skating on thin ice. 1 1 47
200
+ Cynthia nibbled on the carrot . 1 1 31
201
+ Cynthia chewed. 0 1 15
202
+ Paul laughed at Mary. 1 1 21
203
+ Linda winked her lip. 0 1 21
204
+ My heart is pounding me. 0 1 24
205
+ Sharon fainted from hunger. 1 1 27
206
+ The witch poisoned the children. 1 1 32
207
+ A grandfather clock ticked in the hallway. 1 1 42
208
+ I squeaked the door. 0 1 20
209
+ Over the fire there bubbled a fragrant stew. 1 1 44
210
+ Soaring temperatures are predicted for this weekend. 1 1 52
211
+ The fort fluttered with many flags. 0 1 35
212
+ The voices echoed in the hall. 1 1 30
213
+ The stream twists through the valley. 1 1 37
214
+ There presented itself a wonderful opportunity yesterday. 0 1 57
215
+ A wonderful opportunity presented itself to him yesterday. 1 1 58
216
+ Out of the box jumped a little white rabbit. 1 1 44
217
+ Penny skated around the rink. 1 1 29
218
+ Jackie accompanied Rose. 1 1 24
219
+ many information was provided. 0 1 30
220
+ John offers many advice. 0 1 24
221
+ The cake that Jones got was more delicious than the one that Smith got. 1 1 71
222
+ We recommend to eat less cake and pastry. 1 1 41
223
+ I saw that gas can explode. 1 1 27
224
+ He washed her. 1 1 14
225
+ Wash you! 0 1 9
226
+ They have no in. 0 1 16
227
+ Some my jobs are in jeopardy. 0 1 29
228
+ It was the policeman met that several young students in the park last night. 0 1 76
229
+ did the policeman meet in the park? 1 1 35
230
+ John put old books in the box. 1 1 30
231
+ The monkeys proud of their leader. 0 1 34
232
+ John sounded very. 0 1 18
233
+ I don't know if I should agree. 1 1 31
234
+ John bought a book on the table. 1 1 32
235
+ John called the president a fool. 1 1 33
236
+ I forgot how good beer tastes. 1 1 30
237
+ This teacher is a genius. 1 1 25
238
+ A good friend is remained to me by him. 0 1 39
239
+ John ate his noodle quietly. 1 1 28
240
+ A smith hammered the metal. 1 1 27
241
+ Sandy removed her ballet shoes. 1 1 31
242
+ This week will be a difficult one for us. 1 1 41
243
+ Benny worked in a shoe factory when he was a student. 1 1 53
244
+ Everyone hoped that she would sing. 1 1 35
245
+ This proved a decisive factor. 1 1 30
246
+ The crocodile devoured the doughnut. 1 1 36
247
+ John is afraid of Bill. 1 1 23
248
+ John put under the bathtub. 0 1 27
249
+ John placed Kim behind the garage. 1 1 34
250
+ Kim depends for Sandy. 0 1 22
251
+ John put the book in the box. 1 1 29
252
+ John taught English Syntax to new students. 1 1 43
253
+ John regards Bill as a good friend. 1 1 35
254
+ Kim put in the box. 0 1 19
255
+ The chickens seem fond of the farmer. 1 1 37
256
+ The rules require that the executives be polite. 1 1 48
257
+ That Fred was unpopular nominated Bill. 0 1 39
258
+ They were taking a hard look at possible FTA. 1 1 45
259
+ John paid me against the book. 0 1 30
260
+ The committee will study the feasibility of setting up a national computer network. 1 1 83
261
+ We made them be rude. 1 1 21
262
+ After reading the pamphlet, Judy threw it into the garbage can. 1 1 63
263
+ Students studying English reads Conrad's Heart of Darkness while at university. 0 1 79
264
+ I read some of the book. 1 1 24
265
+ Most of the fruit is rotten. 1 1 28
266
+ Neither of students failed. 0 1 27
267
+ I drank some of water. 0 1 22
268
+ It is a golden hair. 1 1 20
269
+ The boy in the doorway waved to his father. 1 1 43
270
+ That dog is so ferocious, it even tried to bite itself. 1 1 55
271
+ He washed yourself. 0 1 19
272
+ John is easy to please Kim. 0 1 27
273
+ There seemed to be intelligent. 0 1 31
274
+ John tried to please Stephen. 1 1 29
275
+ We believed John to be a fountain in the park. 0 1 46
276
+ It tries to leave the country. 0 1 30
277
+ John tries to leave the country. 1 1 32
278
+ Under the bed seems to be a fun place to hide. 1 1 46
279
+ There is believed to be sheep in the park. 0 1 42
280
+ I hope to would study in France. 0 1 32
281
+ John can kick the ball. 1 1 23
282
+ John will rain tomorrow. 0 1 24
283
+ We expect there to will rain. 0 1 29
284
+ John was found in the office. 1 1 29
285
+ Did the child be in the school? 0 1 31
286
+ It did not rain. 1 1 16
287
+ John wants not to leave the town. 1 1 33
288
+ Mary likes to tour art galleries, but Bill hates to. 1 1 52
289
+ They needn't take this exam. 1 1 28
290
+ Ann may spend her vacation in Italy. 1 1 36
291
+ They love to play golf, but I do not. 1 1 37
292
+ George has spent a lot of money, hasn't he? 1 1 43
293
+ As a statesman, scarcely could he do anything worth mentioning. 1 1 63
294
+ We never found any of the unicorns. 1 1 35
295
+ One of Korea's most famous poets wrote these lines. 1 1 51
296
+ Somebody apparently struck the unidentified victim during the early morning hours. 1 1 82
297
+ The car was driven. 1 1 19
298
+ Ricky can be relied on. 1 1 23
299
+ The bed was slept in. 1 1 21
300
+ A pound was weighed by the book. 0 1 32
301
+ Who do you think Tom saw? 1 1 25
302
+ he had spent five thousand dollars. 1 1 35
303
+ Fed knows which politician her to vote for. 0 1 43
304
+ How did you guess that he fixed the computer? 1 1 45
305
+ The committee knows whose efforts to achieve peace the world should honor. 1 1 74
306
+ Which house does your friend live? 0 1 34
307
+ the baker from whom I bought these bagels left. 1 1 47
308
+ I found the place where we can relax. 1 1 37
309
+ Jack is the person with whom Jenny fell in love with. 0 1 53
310
+ There is a bench to sit on. 1 1 27
311
+ I met the man who grows peaches. 1 1 32
312
+ Which topic did you choose without getting his approval? 1 1 56
313
+ Which topic did you get bored because Mary talked about? 0 1 56
314
+ That is the reason why he resigned. 1 1 35
315
+ It bothers me that John coughs. 1 1 31
316
+ To please John is easy. 1 1 23
317
+ Kim is eager to please Tom. 1 1 27
318
+ That we invaded Iraq really bites. 1 1 34
319
+ It annoys me that Fido barks. 1 1 29
320
+ Who achieved the best result was Angela. 0 1 40
321
+ It was the peasant girl who got it. 1 1 35
322
+ That kind of person is hard to find anyone to look after. 1 1 57
323
+ A sick owl doesn't hunt mice. 1 1 29
324
+ Each candidate who has interest in semantics will be admitted to the department. 1 1 80
325
+ Each author whose contribution is written in any language other than English will provide a summary in English. 1 1 111
326
+ I'm sure we even got these tickets! 1 1 35
327
+ I'm even sure we got these tickets! 1 1 35
328
+ It's not because I have any sympathy for urban guerillas that I helped him. 1 1 75
329
+ It isn't because Sue said anything bad about me that I'm angry. 1 1 63
330
+ That he was hungry, John whined. 0 1 32
331
+ 1 gave Mary after the party a book. 0 1 35
332
+ Because she's so pleasant, as for Mary I really like her. 0 1 57
333
+ Though he may hate those that criticize Carter, it doesn't matter. 1 1 66
334
+ With no job would John be happy. 1 1 32
335
+ I have much of the manuscript left to type. 1 1 43
336
+ He's a more reliable man. 1 1 25
337
+ Any trouble is what I don't want. 0 1 33
338
+ They may grow as high as bamboo. 1 1 32
339
+ Some of them made as many errors as 20. 1 1 39
340
+ Sally kissed himself. 0 1 21
341
+ Drew believes I think Rosie loves magazine ads. 1 1 47
342
+ Dave, Dan, Erin, Jaime, and Alina left. 1 1 39
343
+ Hopefully, we'll make it through the winter without snow. 1 1 57
344
+ Blue leather shows herself that Betsy is pretty. 0 1 48
345
+ Gwen hit the baseball. 1 1 22
346
+ I hit that you knew the answer. 0 1 31
347
+ I've lost my wallet or I've lost my mind. 1 1 41
348
+ It was a brand new car that he bought. 1 1 38
349
+ He likes cookies and he hates crumb cake. 1 1 41
350
+ They chased the man with the car. 1 1 33
351
+ I didn't have a red cent. 1 1 25
352
+ the book of poems and from Blackwell takes a very long time to read. 0 1 68
353
+ the one with a red cover takes a very long time to read. 1 1 56
354
+ John has a fear of dogs. 1 1 24
355
+ The building's the roof is leaking. 0 1 35
356
+ the panther's the coat is dark black. 0 1 37
357
+ Colin asked if they could get a mortgage. 1 1 41
358
+ The man I saw get into the cab robbed the bank. 1 1 47
359
+ I know you eat asparagus. 1 1 25
360
+ John's drum will always bother me. 1 1 34
361
+ An evil thought struck Dave. 1 1 28
362
+ In the classroom John put the book on the table. 1 1 48
363
+ Phillip gave the medal to the soldier. 1 1 38
364
+ Heidi thinks that Andy to eat salmon flavored candy bars. 0 1 57
365
+ Heidi thinks that Andy should eat salmon flavored candy bars. 1 1 61
366
+ He danced. 1 1 10
367
+ Did Calvin his homework? 0 1 24
368
+ Sylvia was slapping Jeff upside the head in martial arts class. 1 1 63
369
+ If I am a rich man, I'd buy a diamond ring. 0 1 43
370
+ If he were a rich man, he'd buy a diamond ring. 1 1 47
371
+ John is likely to leave. 1 1 24
372
+ The manager laughed. 1 1 20
373
+ Was sunk. 0 1 9
374
+ Who did you think kissed the gorilla? 1 1 37
375
+ Have you seen my model airplane collection? 1 1 43
376
+ It seems that Lucy was mugged. 1 1 30
377
+ I ate a salad that was filled with lima beans. 1 1 46
378
+ There were four men arriving at the station when I pulled up. 1 1 61
379
+ Jean is reluctant to dance. 1 1 27
380
+ I want her to dance. 1 1 20
381
+ Jean persuaded Robert. 1 1 22
382
+ Jean wants Bill to do the Macarena. 1 1 35
383
+ The children admire their mother. 1 1 33
384
+ Who has Peter talked with? 1 1 26
385
+ Heidi likes her violin. 1 1 23
386
+ John thinks that Mary loves himself. 0 1 36
387
+ Which pictures of himself does John like? 1 1 41
388
+ Any owl hunts mice. 1 1 19
389
+ Any man didn't eat dinner. 0 1 26
390
+ A pilot could be flying this plane. 1 1 35
391
+ Everybody who attended last week's huge rally, whoever they were, signed the petition. 1 1 86
392
+ Anybody who attended last week's huge rally signed the petition. 1 1 64
393
+ Any tiger has orange fur, marked with black stripes. 1 1 52
394
+ Any albino tiger has orange fur, marked with black stripes. 1 1 59
395
+ You must pick any flower you see. 1 1 33
396
+ Any pilot on duty today could be flying this plane. 1 1 51
397
+ You may pick every flower, but leave a few for Mary. 0 1 52
398
+ The Dodgers beat the Red Sox and were beaten by the Giants. 1 1 59
399
+ In which car was the man seen? 1 1 30
400
+ The man who Mary loves and Sally hates computed my tax. 1 1 55
401
+ The kennel which Mary made and Fido sleeps has been stolen. 0 1 59
402
+ Tom said he would and Bill actually did eat a raw eggplant. 1 1 59
403
+ The wealthy young man bought that piano for his secret fiancée. 1 1 63
404
+ The dog stole the turkey. 1 1 25
405
+ a tall building. 1 0 16
406
+ This building is tall. 1 1 22
407
+ I like the book which you gave me. 1 1 34
408
+ Captain Wentworth wrote a letter to Anne Elliott. 1 1 49
409
+ She asked was Alison coming to the party. 1 1 41
410
+ They realised that never had Sir Thomas been so offended. 1 1 57
411
+ Fanny regretted having to talk to Aunt Norris. 1 1 46
412
+ Knowing the country well, he took a short cut. 1 1 46
413
+ He left the train with somebody else's wallet in his pocket. 1 1 60
414
+ Magnus went to Ireland. 1 1 23
415
+ Who did John send the book? 1 1 27
416
+ The idea dismayed the Prime Minister that the Dome was dull. 1 1 60
417
+ Came right in he did without so much as a knock. 1 1 48
418
+ Harriet admired Mr Knightley. 1 1 29
419
+ The book is boring. 1 1 19
420
+ Ethel wishes to ask you some awkward questions. 1 1 47
421
+ Emma made Harriet some food. 1 1 28
422
+ The window was broken with a hammer. 1 1 36
423
+ The fugitive lay motionless in order to avoid discovery. 1 1 56
424
+ The guard marched the prisoners round the yard. 1 1 47
425
+ Frank Churchill crossed the street. 1 1 35
426
+ Emma and Harriet were attacked yesterday. 1 1 41
427
+ Fiona might be here by 5 o'clock. 1 1 33
428
+ I am both expecting to get the job and of the opinion that it is a desirable one. 1 1 81
429
+ Pat was awarded the Golden Fleece Award and very upset about it. 1 1 64
430
+ Kim alienated cats and beating his dog. 0 1 39
431
+ They knew that pictures of each other would be on sale. 1 1 55
432
+ Which article did Terry file papers without reading? 0 1 52
433
+ I want to try and buy some whiskey. 1 1 35
434
+ She goes and buying some whiskey. 0 1 33
435
+ Kim and Terry is happy. 0 1 23
436
+ Kim gave a dollar to Bobbie and a dime to Jean. 1 1 47
437
+ This girl in the red coat will put a picture of Bill in the mailbox and on your desk before tomorrow. 1 1 101
438
+ Put a picture of Bill on your desk before tomorrow, this girl in the red coat will put a picture of Bill on your desk before tomorrow. 0 1 134
439
+ Picture of Bill, this girl in the red coat will put a picture of Bill on your desk before tomorrow. 0 1 99
440
+ Ann bought a first edition of Richard III for $1000. 1 1 52
441
+ John became deadly afraid of flying. 1 1 36
442
+ What John promised is to be gentle. 1 1 35
443
+ Henri wants to buy which books about cooking? 1 1 45
444
+ I will fix the computer for Carol. 1 1 34
445
+ Smith loaned a valuable collection of manuscripts to the library. 1 1 65
446
+ The paper is cut by Mary. 1 1 25
447
+ It was from six to nine or it took place from six to nine. 1 1 58
448
+ Suddenly, there arrived two inspectors from the INS. 1 1 52
449
+ The soup cooks, thickens. 1 1 25
450
+ I gave money to the charity. 1 1 28
451
+ John asked. 1 1 11
452
+ I saw even the student. 1 1 23
453
+ John whispered Mary left. 0 1 25
454
+ Mary wonders that Bill will come. 0 1 33
455
+ Sophie will theater. 0 1 20
456
+ John finished the cake and drank the lemonade. 1 1 46
457
+ Herself likes Mary's mother. 0 1 28
458
+ Each of the boys fought with some of the other boys. 1 1 52
459
+ John's mother likes himself. 0 1 28
460
+ Mary revealed himself to John. 0 1 30
461
+ Mary believes that Bill saw himself. 1 1 36
462
+ John heard that they criticized themselves. 0 1 43
463
+ Mary thinks that she is smart. 1 1 30
464
+ Henry found that Bill is sad. 1 1 29
465
+ John considers himself proud of Mary. 1 1 37
466
+ You should sit before there. 0 1 28
467
+ There is a nurse available. 1 1 27
468
+ Everyone hopes everyone to sleep. 0 1 33
469
+ Everyone hopes that he will sleep. 1 1 34
470
+ Only Churchill remembered Churchill giving the Blood, Sweat and Tears speech. 0 1 77
471
+ John promise Mary to shave himself. 1 1 35
472
+ John convinced Bill that Mary should sleep. 1 1 43
473
+ I saw Bill while you did so Mary. 0 1 33
474
+ What Mary did Bill was give a book. 0 1 35
475
+ The bears sniffed 1 1 17
476
+ to steal talismans from witches is dangerous 1 1 44
477
+ Anson demonized 0 1 15
478
+ You kicked yourselves 1 1 21
479
+ The book that I said that I'd never read. 1 1 41
480
+ Anson demonized up the Khyber 0 1 29
481
+ It is not true that I might be going to the party. 1 1 50
482
+ Kick you! 0 1 9
483
+ Jason persuaded the doctor to treat Medea 1 1 41
484
+ Who did you get an accurate description of? 1 1 43
485
+ The Peter we all like was at the party 1 1 38
486
+ The government's imposition of a fine. 1 1 38
487
+ Did Athena help us? 1 1 19
488
+ You wanted to meet the man I saw. 1 1 33
489
+ Anson demonized David every day 1 1 31
490
+ There is a programme about Euripides on a Radio 4 tonight? 1 1 58
491
+ Medea denied poisoning the phoenix. 1 1 35
492
+ The elixir was hidden in a hole in the ground 1 1 45
493
+ Extremely frantically, Anson danced at Trade 1 1 44
494
+ What did you ask who Medea gave? 0 1 32
495
+ It's Anson that I like 1 1 22
496
+ Medea tried the nurse to poison her children. 0 1 45
497
+ I felt that I know you. 1 1 23
498
+ Will he can do it? 1 1 18
499
+ I ate that she was happy 0 1 24
500
+ Anson believed himself to be handsome. 1 1 38
501
+ We arrived about six o'clock 1 1 28
502
+ Gilgamesh is having fought the dragon. 0 1 38
503
+ Anson thought that himself was happy. 0 1 37
504
+ Fortunately Hermione passed biology 1 1 35
505
+ I might be not going to the party but washing my hair 1 1 53
506
+ I thought he liked myself 0 1 25
507
+ The butler sent the poison to Dinah. 1 1 36
508
+ Jason happens to appear to seem to be sick. 1 1 43
509
+ What did you ask who saw? 0 1 25
510
+ We linguists love to argue 1 1 26
511
+ He can will go 0 1 14
512
+ How fierce the battle? 0 1 22
513
+ Which king did you ask which city invaded? 0 1 42
514
+ It is some disgruntled old pigs in those ditches that humans love to eat. 1 1 73
515
+ That banana is eating the monkey. 1 1 33
516
+ Burn them! 1 1 10
517
+ It stinks that Aphrodite is omnipotent. 1 1 39
518
+ Agamemnon seems to be a maniac 1 1 30
519
+ Will Anson come to the party? 1 1 29
520
+ I wondered who Medea had poisoned. 1 1 34
521
+ I inquired when could we leave. 0 1 31
522
+ They kicked them 1 1 16
523
+ The monkey is ate the banana 0 1 28
524
+ I would like to could swim 1 1 26
525
+ I kicked myself 1 1 15
526
+ The bookcase ran 0 1 16
527
+ I shaved myself. 1 1 16
528
+ Anson became a muscle bound. 1 1 28
529
+ Somebody just left - guess who. 1 1 31
530
+ They claimed they had settled on something, but it wasn't clear what they had settled on. 1 1 89
531
+ If Sam was going, Sally would know where. 1 1 41
532
+ They're going to serve the guests something, but it's unclear what. 1 1 67
533
+ She's reading. I can't imagine what. 1 1 36
534
+ John said Joan saw someone from her graduating class. 1 1 53
535
+ John ate dinner but I don't know who. 0 1 37
536
+ She mailed John a letter, but I don't know to whom. 0 1 51
537
+ I served leek soup to my guests. 1 1 32
538
+ I served my guests. 1 1 19
539
+ She was bathing, but I couldn't make out who. 0 1 45
540
+ She knew French for Tom. 0 1 24
541
+ John is tall on several occasions. 0 1 34
542
+ The ship sank, but I don't know with what. 0 1 42
543
+ They noticed the painting, but I don't know for how long. 0 1 57
544
+ John was tall, but I don't know on what occasions. 0 1 50
545
+ Joan ate dinner with someone but I don't know who. 1 1 50
546
+ Joan ate dinner with someone but I don't know who with. 1 1 55
547
+ I know that Meg's attracted to Harry, but they don't know who. 0 1 62
548
+ Since Jill said Joe had invited Sue, we didn't have to ask who. 0 1 63
549
+ I know that Meg's attracted to Harry, but they don't know who. 1 1 62
550
+ She said she had spoken to everybody, but he wasn't sure who. 0 1 61
551
+ Each of the performers came in, but were sitting so far back that we couldn't see who. 0 1 86
552
+ She didn't talk to one student. 1 1 31
553
+ She doesn't meet anyone for dinner, but they can't figure out who. 0 1 66
554
+ Everyone relies on someone. It's unclear who. 1 1 45
555
+ Each student wrote a paper on a Mayan language, but I don't remember which one. 1 1 79
556
+ The newspaper has reported that they are about to appoint someone, but I can't remember who the newspaper has reported that they are about to appoint. 1 1 150
557
+ The newspaper has reported that they are about to appoint someone, but I can't remember who they are about to appoint. 1 1 118
558
+ Most columnists claim that a senior White House official has been briefing them, and the newspaper today reveals which one. 1 1 123
559
+ Most columnists claim that a senior White House official has been briefing them, but none will reveal which one. 1 1 112
560
+ Bill wondered how many papers Sandy had read, but he didn't care which ones. 1 1 76
561
+ I never know which papers Sandy has read, but I usually know how many. 1 1 70
562
+ Sandy had read how many papers?! 1 1 32
563
+ Everybody gets on well with a certain relative, but often only his therapist knows which one. 1 1 93
564
+ Which book did each author recommend? 1 1 37
565
+ His or her least known work. 1 1 28
566
+ They were going to meet sometime on Sunday, but the faculty didn't know when. 1 1 77
567
+ John likes some students, but I don't know who. 1 1 47
568
+ I don't know who John likes. 1 1 28
569
+ John likes some students, but I don't know who John likes some students. 0 1 72
570
+ Joan said she talked to the students, but Fred couldn't figure out who. 0 1 71
571
+ He announced he had eaten the asparagus, but we didn't know what. 0 1 65
572
+ She was reading the books under the table, but Fred didn't know what books. 1 1 75
573
+ He announced he would marry the woman he loved most, but none of his relatives could figure out who. 1 1 100
574
+ She talked to John or Mary but I don't know which. 1 1 50
575
+ She talked to John or Mary but I don't know which one. 1 1 54
576
+ She talked to Harry, but I don't know who else. 1 1 47
577
+ I will see them, but I don't know how many of them. 1 1 51
578
+ Everyone who knows either Susan or Laura likes her. 1 1 51
579
+ She said she talked to three students but I don't know how many. 0 1 64
580
+ She said she talked to those students but I don't know how many. 0 1 64
581
+ He shouted again, but I don't know who to. 1 1 42
582
+ She was dancing with somebody, but I don't know who with. 1 1 57
583
+ Several firefighters were injured, but it's not known. 1 1 54
584
+ Meg is attracted to Harry, but they don't know who she is attracted to. 1 1 71
585
+ Sandy was trying to work out which students would be able to solve a certain problem, but she wouldn't tell us which one. 1 1 121
586
+ Sandy was trying to work out which students would be able to solve a certain problem, but she wouldn't tell us which one. 0 1 121
587
+ John and someone were dancing together, but I don't know who. 0 1 61
588
+ The TA's have been arguing about whether some student or other should pass, but I can't now remember which one. 1 1 111
589
+ It has been determined that somebody will be appointed; it's just not clear yet who. 0 1 84
590
+ Sally asked if somebody was going to fail math class, but I can't remember who. 0 1 79
591
+ The TA's have been arguing about whether some student or other should pass, but I can't now remember which one. 0 1 111
592
+ Sandy is very anxious to see if the students will be able to solve the homework problem in a particular way, but she won't tell us which. 1 1 137
593
+ Sandy is very anxious to see if the students will be able to solve the homework problem in a particular way, but she won't tell us in which way. 1 1 144
594
+ Clinton is anxious to find out which budget dilemmas Panetta would be willing to tackle in a certain way, but he won't say in which. 1 1 132
595
+ Sandy is wondering whether there will be students who have to drop the class for a certain reason, but she won't reveal what. 1 1 125
596
+ In which way is Sandy very anxious to see if the students will be able to solve the homework problem? 0 1 101
597
+ In which way is Clinton anxious to find out which budget dilemmas Panetta would be willing to solve? 0 1 100
598
+ I know how many assignments I've graded, but I don't know how many Bill has. 1 1 76
599
+ What did you leave before they did? 0 1 35
600
+ What did you leave before they started playing? 0 1 47
601
+ Sandy was trying to work out which students would be able to solve a certain problem. 1 1 85
602
+ The administration has issued a statement that it is willing to meet with one of the student groups. 1 1 100
603
+ Sandy was trying to work out which students would be able to solve a problem. 1 1 77
604
+ The administration has issued a statement that it is willing to meet a student group. 1 1 85
605
+ The administration has issued a statement that it is willing to meet a student group, but I'm not sure which one. 1 0 113
606
+ I think Agnes said that Bill would speak, but I don't remember what about. 1 1 74
607
+ Agnes wondered how John could eat but it's not clear what. 0 1 58
608
+ Tony sent Mo a picture that he painted, but it's not clear with what. 0 1 69
609
+ She's been dancing but we don't know with whom. 1 1 47
610
+ Who did they see someone? 0 1 25
611
+ It was believed by everybody that Mary was a thief. 1 1 51
612
+ That professor is feared by all students. 1 1 41
613
+ Mary was respected by John. 1 1 27
614
+ Ted was bitten by the spider. 1 1 29
615
+ The book was by John written. 0 1 29
616
+ The argument was summed by the coach up. 0 1 40
617
+ The paper was written up by John. 1 1 33
618
+ The paper was written by John up. 0 1 33
619
+ John was spoken to by Mary. 1 1 27
620
+ John was spoken by Mary to. 0 1 27
621
+ The book was seen by Mary. 1 1 26
622
+ John was seen the book. 0 1 23
623
+ The book was written. 1 1 21
624
+ John was spoke by Mary to. 0 1 26
625
+ The table was wiped clean by John. 1 1 34
626
+ The table was wiped by John clean. 0 1 34
627
+ Mary was given by John the book. 0 1 32
628
+ John was believed to be telling the truth by Mary. 1 1 50
629
+ John was believed by Mary to be telling the truth. 1 1 50
630
+ The car was driven by John to Maine. 1 1 36
631
+ It was believed by the students that they would have an exam. 1 1 61
632
+ The magazines were sent to herself by Mary. 0 1 43
633
+ Chocolate eggs were hidden from each other by the children. 0 1 59
634
+ The magazines were sent by Mary to herself. 1 1 43
635
+ Chocolate eggs were hidden from no child by any adult. 1 1 54
636
+ Tabs were kept on each agent by the other. 1 1 42
637
+ Chocolate eggs were hidden from every child by his mother. 1 1 58
638
+ Books were taken from no student and given to Mary. 1 1 51
639
+ Books were taken from no student and given to Mary by any professor. 0 1 68
640
+ Books were taken from each student by the other. 1 1 48
641
+ Books were taken from each student and given to Mary. 1 1 53
642
+ Books were taken from each student and given to Mary by the other. 0 1 66
643
+ Jack hates Sue and is loved by Mary. 1 1 36
644
+ Vera sent a baby alligator to Max and a leather dinosaur to Phyllis. 1 1 68
645
+ Either Sam plays the bassoon or Jekyll the oboe. 1 1 48
646
+ Sam doesn't play bassoon, nor Medusa oboe. 1 1 42
647
+ Bill ate the peaches, but Harry the grapes. 0 1 43
648
+ I no more could have stolen that steak than Jack the diamonds. 1 1 62
649
+ Bill ate more peaches than Harry did grapes. 1 1 44
650
+ Bill ate the peaches and Harry did the grapes. 0 1 46
651
+ Tom will smoke the grass, and Reuben has the hash. 0 1 50
652
+ If the ants were called elephants and elephants ants, I'd be able to squash an elephant. 1 1 88
653
+ Simon quickly dropped the gold, and Jack the diamonds. 1 1 54
654
+ Bob tried to wash himself, and Mary to read the funnies. 1 1 56
655
+ Harry told Sue that Albania is a lovely place for a vacation, and Tom told Sally that Albania is a lovely place for a vacation. 1 1 127
656
+ Harry told Sue that Albania is a lovely place for a vacation, and Tom. 1 1 70
657
+ Max seemed to be trying to begin to love Harriet, and Fred to be trying to begin to love Sue. 1 1 93
658
+ Max seemed to be trying to force Ted to leave the room, and Walt, Ira. 1 1 70
659
+ Max seemed to be trying to force Ted to leave the room, and Walt to stay a little longer. 0 1 89
660
+ Arizona elected Goldwater Senator, and Massachusetts, McCormack. 0 1 64
661
+ Millie will send the President an obscene telegram, and Paul, the Secretary a rude letter. 0 1 90
662
+ Maytag will give a brand-new dryer to the winner of the Mrs. 0 1 60
663
+ Bill didn't eat the peaches, nor Harry. 0 1 39
664
+ Bill ate the peaches, and Harry did, too. 1 1 41
665
+ Bill must quickly eat the peaches, and Harry must slowly. 0 1 57
666
+ Whenever Russia has made a major political blunder, the U.S. has too. 1 1 69
667
+ Bill's story about Sue and Max's about Kathy both amazed me. 1 1 60
668
+ I bought three quarts of wine and two of Clorox. 1 1 48
669
+ Scientists at the South Hanoi Institute of Technology have succeeded in raising one dog with five legs, another with a cow's liver, and a third with no head. 1 0 157
670
+ Bill's story about Sue may be amazing, but Max's is virtually incredible. 1 1 73
671
+ I like Bill's yellow shirt, but not Max's. 1 1 42
672
+ Bill's funny story about Sue and Max's boring one about Kathy both amazed me. 1 1 77
673
+ Bill's wine from France and Ted's from California cannot be compared. 1 1 69
674
+ As a teacher, you have to deal simultaneously with the administration's pressure on you to succeed, and the children's to be a nice guy. 0 1 136
675
+ Neither Von Karajan's recording of Beethoven's 6th on Columbia nor Klemperer's on Angel has the right tempo. 1 1 108
676
+ Gould's performance of Bach on the piano doesn't please me anywhere as much as Ross's on the harpsichord. 0 1 105
677
+ Tom's dog with one eye attacked Frank's with three legs. 0 1 56
678
+ Because Steve's of a spider's eye had been stolen, I borrowed Fred's diagram of a snake's fang. 0 1 95
679
+ Neither von Karajan's recording of Beethoven's 6th on Columbia nor Klemperer's has the right tempo. 1 1 99
680
+ Tom's dog with one eye attacked Fred's. 1 1 39
681
+ I borrowed Fred's diagram of a snake's eye because Steve's had been stolen. 1 1 75
682
+ Jerry attempted to blow up the Pentagon. 1 1 40
683
+ So fast did he run that nobody could catch him. 1 1 47
684
+ Bill bought a red house, and Max bought one too. 1 1 48
685
+ Who always drinks milk? 1 1 23
686
+ The book which inspired them was very long. 1 1 43
687
+ The book what inspired them was very long. 0 1 42
688
+ I know the person whose mother died. 1 1 36
689
+ The person whose mother's dog we were all fond of. 1 1 50
690
+ I wonder whose mother died. 1 1 27
691
+ I wonder whose mother's dog died. 1 1 33
692
+ I wonder to whom they dedicated the building. 1 1 45
693
+ Give me the phone number of the person whose mother's dog died. 1 1 63
694
+ This is the senator to whose mother's friend's sister's I sent the letter. 1 1 74
695
+ I want goes to the store. 0 1 25
696
+ I wonder what to be a clown on the cover of. 0 1 44
697
+ Bother you that Kim left! 0 1 25
698
+ A student who to talk to us just walked in. 0 1 43
699
+ Whose bagels do you like? 1 1 25
700
+ I wonder in whom to place my trust. 1 1 35
701
+ There were several old rocks songs that she and I were the only two who knew. 1 1 77
702
+ It was to to amuse us that Kim was singing that they wanted. 0 1 60
703
+ What they feared most was to be no one available to help them. 0 1 62
704
+ We tried to amuse them that Kim was singing. 0 1 44
705
+ Mary asked me if, in St. Louis, John could rent a house cheap. 1 1 62
706
+ Mary arranged for, in St. Louis, John to rent a house cheap. 0 1 60
707
+ It would be unwise for there to be no fire exit. 1 1 48
708
+ I believe there to be no way out. 1 1 33
709
+ I wonder in whom them to place their trust. 0 1 43
710
+ I wonder whom us to trust. 0 1 26
711
+ I wonder who for us to trust. 0 1 29
712
+ I wonder who to place my trust in. 1 1 34
713
+ I know the people that voted in the election. 1 1 45
714
+ I threw away a book that Sandy thought we had read. 1 1 51
715
+ I thought that you were sick. 1 1 29
716
+ I dislike the people in who we placed our trust. 0 1 48
717
+ I dislike the company in which we placed our trust. 1 1 51
718
+ I dislike the people in whose house we stayed. 1 1 46
719
+ I dislike the person with whom we were talking. 1 1 47
720
+ Jones, that we were talking to last night, always watches football games alone. 0 1 79
721
+ A letter was received that Jones would be upset by. 1 1 51
722
+ A letter was received Jones would be upset by. 0 1 46
723
+ I saw someone yesterday I hadn't seen for years. 1 1 48
724
+ Something happened I couldn't really talk about. 1 1 48
725
+ The only person that I like whose kids Dana is willing to put up with is Pat. 1 1 77
726
+ The book that I like which everyone else in the class hates was written by John. 1 1 80
727
+ The only person whose kids Dana is willing to put up with was written by John. 0 1 78
728
+ The book that I like - everyone else in the class hates. 0 1 56
729
+ The only person whose kids Dana is willing to put up with is Pat. 1 1 65
730
+ Which book's, author did you meet? 1 1 34
731
+ Which boy's, mother, did you meet who you liked? 0 1 48
732
+ Which book's, author did you meet who you liked? 0 1 48
733
+ Which boy's, mother, did you meet? 1 1 34
734
+ All who lost money in the scam are eligible for the program. 1 1 60
735
+ Who for Sandy to talk to is still enrolled in the class? 0 1 56
736
+ Who who you like does Sandy also like? 1 1 38
737
+ Everything you like is on the table. 1 1 36
738
+ The bills passed by the House yesterday that we objected to were vetoed. 1 1 72
739
+ The only people being added to our group who were at Harvard were students. 1 1 75
740
+ We like ourselves. 1 1 18
741
+ Nobody likes us. 1 1 16
742
+ Leslie likes ourselves. 0 1 23
743
+ Ourselves like ourselves. 0 1 25
744
+ She voted for herself. 1 1 22
745
+ We gave us presents. 0 1 20
746
+ We gave ourselves presents. 1 1 27
747
+ We gave presents to ourselves. 1 1 30
748
+ We gave us to the cause. 0 1 24
749
+ We gave ourselves to the cause. 1 1 31
750
+ Leslie told us about us. 0 1 24
751
+ Leslie told ourselves about us. 0 1 31
752
+ We think that Leslie likes us. 1 1 30
753
+ We think that Leslie likes ourselves. 0 1 37
754
+ Our friends like us. 1 1 20
755
+ Those pictures of us offended us. 1 1 33
756
+ We found your letter to ourselves in the trash. 0 1 47
757
+ Vote for you! 0 1 13
758
+ Vote for yourself! 1 1 18
759
+ We appeared to them to vote for themselves. 0 1 43
760
+ We admired the pictures of us in the album. 1 1 43
761
+ We admired the pictures of ourselves in the album. 1 1 50
762
+ Leslie used a pen. 1 1 18
763
+ We put the pigs in a pen. 1 1 25
764
+ We need to pen the pigs to keep them from getting into the corn. 1 1 64
765
+ They should pen the letter quickly. 1 1 35
766
+ The car won't run. 1 1 18
767
+ This dye will run. 1 1 18
768
+ She can run an accelerator. 1 1 27
769
+ These stockings will run. 1 1 25
770
+ We need another run to win. 1 1 27
771
+ Lee saw the student with a telescope. 1 1 37
772
+ I forgot how good beer tastes. 1 1 30
773
+ Visiting relatives can be boring. 1 1 33
774
+ If only Superman would stop flying planes! 1 1 42
775
+ That's a new car dealership. 1 1 28
776
+ I know you like the back of my hand. 1 1 36
777
+ Max is on the phone now. 1 1 24
778
+ I saw her duck. 1 1 15
779
+ I'm creating a committee. Kim – you're in charge. 1 1 49
780
+ Lights go out at ten. There will be no talking afterwards. 1 1 58
781
+ They found the book on the atom. 1 1 32
782
+ Which experts testified against defendants who exposed them? 1 1 60
783
+ List all experts for the defendant who represented himself. 1 1 59
784
+ List associates of each defendant who speaks Spanish. 1 1 53
785
+ They lost themselves' books. 0 1 28
786
+ Some sentences go on and on and on. 1 1 35
787
+ Sentences some go on and on and on and on. 0 1 42
788
+ That surprised me. 1 1 18
789
+ I noticed the. 0 1 14
790
+ They were interested in his. 1 1 28
791
+ This is my favorite. 1 1 20
792
+ A large dog chased a small cat. 1 1 31
793
+ Some people yell at noisy dogs in my neighborhood. 1 1 50
794
+ Some people yell at the dogs in my neighborhood. 1 1 48
795
+ Some people yell at the dogs. 1 1 29
796
+ Some people yell at noisy dogs. 1 1 31
797
+ Some people yell at dogs. 1 1 25
798
+ Some people consider the noisy dogs dangerous. 1 1 46
799
+ Some people consider the dogs in my neighborhood dangerous. 1 1 59
800
+ Some people consider noisy dogs in my neighborhood dangerous. 1 1 61
801
+ Some people consider the dogs dangerous. 1 1 40
802
+ Some people consider noisy dogs dangerous. 1 1 42
803
+ Some people consider dogs in my neighborhood dangerous. 1 1 55
804
+ Some people consider dogs dangerous. 1 1 36
805
+ People with children who use drugs should be locked up. 1 1 55
806
+ This disease gave Leslie a fever in Rome. 1 1 41
807
+ The love of my life and mother of my children would never do such a thing. 1 1 74
808
+ Most elections are quickly forgotten, but the election of 2000, everyone will remember for a long time. 1 1 103
809
+ It is painting by Klee or drawing by Miro that the museum displays no. 0 1 70
810
+ The defendant denied the accusation. 1 1 36
811
+ The teacher disappeared the problem. 0 1 36
812
+ The teacher handed the student. 0 1 31
813
+ The bird sings. 1 1 15
814
+ The bird sing. 0 1 14
815
+ Birds sings. 0 1 12
816
+ The birds give the worm a tug. 1 1 30
817
+ The bird give the worm a tug. 0 1 29
818
+ The birds gives the worm a tug. 0 1 31
819
+ Terry delighted in my pain. 1 1 27
820
+ Terry delighted. 0 1 16
821
+ Terry delighted my pain. 0 1 24
822
+ Kerry remarked it was late. 1 1 27
823
+ What additional categories and rules would be required to handle these verbs? 1 1 77
824
+ We created a monster. 1 1 21
825
+ I was already aware of fact. 0 1 28
826
+ The defendant deny the allegation. 0 1 34
827
+ The defendants denies the allegation. 0 1 37
828
+ The defendant walks. 1 1 20
829
+ The defendant walk. 0 1 19
830
+ The defendants walks. 0 1 21
831
+ How many feature structures categories can label the first daughter? 1 1 68
832
+ The child put the toy on the table. 1 1 35
833
+ The teacher became angry with the students. 1 1 43
834
+ The teacher became. 0 1 19
835
+ The jury believed the defendant lied. 1 1 37
836
+ The guests dined. 1 1 17
837
+ We relied on Leslie. 1 1 20
838
+ We relied above Leslie. 0 1 23
839
+ We celebrated in the streets. 1 1 29
840
+ We celebrated in the streets in the rain on Tuesday in the morning. 1 1 67
841
+ The children are happy of ice cream. 0 1 36
842
+ The children are fond with the ice cream. 0 1 41
843
+ The children are fond that they have ice cream. 0 1 47
844
+ A magazine appeared on the newsstands. 1 1 38
845
+ A magazine about crime appeared on the newsstands. 1 1 50
846
+ Newsweek appeared on the newsstands. 1 1 36
847
+ Newsweek about crime appeared on the newsstands. 0 1 48
848
+ The report that crime was declining surprised many people. 1 1 58
849
+ The book surprised many people. 1 1 31
850
+ The book that crime was declining surprised many people. 0 1 56
851
+ The storm arrived after the picnic. 1 1 35
852
+ The storm arrived while the picnic. 0 1 35
853
+ The storm arrived while we ate lunch. 1 1 37
854
+ This dogs barked. 0 1 17
855
+ These dogs barked. 1 1 18
856
+ A chair was broken. 1 1 19
857
+ They want them arrested. 1 1 24
858
+ They preferred them arrested. 1 1 29
859
+ We preferred them on our team. 1 1 30
860
+ With my parents as supportive as they are, I'll be in fine shape. 1 1 65
861
+ We walks. 0 1 9
862
+ Few dog barked. 0 1 15
863
+ The dogs barked. 1 1 16
864
+ I walk and Dana runs. 1 1 21
865
+ They like us. 1 1 13
866
+ Us like them. 0 1 13
867
+ Kim likes dogs. 1 1 15
868
+ Dogs like Kim. 1 1 14
869
+ The person responsible confessed. 1 1 33
870
+ The person confessed responsible. 0 1 33
871
+ The cat slept soundly and furry. 0 1 32
872
+ The soundly and furry cat slept. 0 1 32
873
+ Chris walks, Pat eats broccoli, and Sandy plays squash. 1 1 55
874
+ There was some particular dog who saved every family. 1 1 53
875
+ Susan frightens her. 1 1 20
876
+ Susan told her a story. 1 1 23
877
+ Susan told a story to her. 1 1 26
878
+ Susan devoted herself to linguistics. 1 1 37
879
+ Nobody told Susan about herself. 1 1 32
880
+ That picture of Susan offended her. 1 1 35
881
+ He offended Sandy. 1 1 18
882
+ I enjoy yourself. 0 1 17
883
+ They talk to themselves. 1 1 24
884
+ Nobody told Susan. 1 1 18
885
+ Protect yourself! 1 1 17
886
+ Protect you! 0 1 12
887
+ I met the person who left. 1 1 26
888
+ Leslie slept. 1 1 13
889
+ Chris handed Bo. 0 1 16
890
+ Dana walked and Leslie ran. 1 1 27
891
+ Dana walking and Leslie ran. 0 1 28
892
+ Dana walking and Leslie running. 0 1 32
893
+ the putter of books left. 0 1 25
894
+ Kris donated a book to the library. 1 1 35
895
+ The police sprayed the protesters with water. 1 1 45
896
+ The police sprayed water on the protesters. 1 1 43
897
+ The students drove cars. 1 1 24
898
+ These cars drive easily. 1 1 24
899
+ The horse kicked me black and blue. 1 1 35
900
+ They yelled. 1 1 12
901
+ The horse raced past the barn fell. 1 1 35
902
+ The horse that was raced past the barn fell. 1 1 44
903
+ The boat seen down the river sank. 1 1 34
904
+ The evidence assembled by the prosecution convinced the jury. 1 1 61
905
+ Lou forgot the umbrella. 1 1 24
906
+ Lou forgot the umbrella in the closet. 1 1 38
907
+ Lou hoped the umbrella was broken. 1 1 34
908
+ Lou hoped the umbrella in the closet. 0 1 37
909
+ Lou put the umbrella was broken. 0 1 32
910
+ Lou put the umbrella in the closet. 1 1 35
911
+ The artist drew the child with a pencil. 1 1 40
912
+ The dog bit the cat. 1 1 20
913
+ The cat was bitten the mouse. 0 1 29
914
+ The cat was bitten the mouse by the dog. 0 1 40
915
+ Chris was handed Sandy a note by Pat. 0 1 37
916
+ Chris was handed a note. 1 1 24
917
+ Chris was handed Sandy a note. 0 1 30
918
+ TV puts dumb ideas in children's heads. 1 1 39
919
+ Dumb ideas are put in children's heads by TV. 1 1 45
920
+ Dumb ideas are put in children's heads. 1 1 39
921
+ Dumb ideas are put notions in children's heads by TV. 0 1 53
922
+ The patient died. 1 1 17
923
+ The patient was died. 0 1 21
924
+ Chris was handed. 0 1 17
925
+ TV puts dumb ideas. 0 1 19
926
+ He was arrested by the police. 1 1 30
927
+ The cat got bitten. 1 1 19
928
+ The cat were bitten by the dog. 0 1 31
929
+ There is a monster in Loch Ness. 1 1 32
930
+ It is obvious that Pat is lying. 1 1 32
931
+ Pat is the captain of the team. 1 1 31
932
+ Pat is hate Chris. 0 1 18
933
+ There is a unicorn in the garden. 1 1 33
934
+ There was a felon elected to the city council. 1 1 46
935
+ There is a seat available. 1 1 26
936
+ A seat available was in the last row. 0 1 37
937
+ Many people were fond of Pat. 1 1 29
938
+ People are looking through the window. 1 1 38
939
+ A felon was elected to the city council. 1 1 40
940
+ There loved Sandy. 0 1 18
941
+ We talked to them about there. 0 1 30
942
+ It mattered that the Giants had lost. 1 1 37
943
+ That dogs bark annoys people. 1 1 29
944
+ It annoys people that dogs bark. 1 1 32
945
+ That Chris knew the answer occurred to Pat. 1 1 43
946
+ It never occurred to Pat that Chris knew the answer. 1 1 52
947
+ That the Cardinal won the game gave Sandy a thrill. 1 1 51
948
+ It gave Sandy a thrill that the Cardinal won the game. 1 1 54
949
+ That Sandy had lied suggested. 0 1 30
950
+ It loved Sandy. 0 1 15
951
+ Cohen proved the independence of the continuum hypothesis. 1 1 58
952
+ Cohen proved that the continuum hypothesis was independent. 1 1 59
953
+ We forgot our invitations. 1 1 26
954
+ Nobody saw Pat. 1 1 15
955
+ That Fido barks annoys me. 1 1 26
956
+ Fido barks. 1 1 11
957
+ Chris dreads the bucket. 1 1 24
958
+ The candidates bring advantage to the voters. 1 1 45
959
+ Tabs are kept on suspected drug dealers by the FBI. 1 1 51
960
+ Advantage is taken of every opportunity for improvement. 1 1 56
961
+ The bucket was kicked by Pat. 1 1 29
962
+ John is sad. 1 1 12
963
+ John loaded the wagon full with hay. 1 1 36
964
+ John loaded the wagon with hay green. 0 1 37
965
+ I presented John with it dead. 0 1 30
966
+ Of whom are you thinking? 1 1 25
967
+ John became rich. 1 1 17
968
+ I gave John gold apples. 1 1 24
969
+ how silly is Bill considered? 1 1 29
970
+ how mad was Bill made? 1 1 22
971
+ John is sick. 1 1 13
972
+ John left singing. 1 1 18
973
+ John is near Larry. 1 1 19
974
+ John gave Bill the dog dead. 1 1 28
975
+ Bill was struck by John as stupid. 0 1 34
976
+ John was struck as sick. 0 1 24
977
+ John was struck by Bill's idiocy. 1 1 33
978
+ John promised Bill to leave. 1 1 28
979
+ John tried to leave. 1 1 20
980
+ To leave would be a pleasure. 1 1 29
981
+ John was struck by Bill as pompous. 0 1 35
982
+ John was promised by Bill to leave. 0 1 35
983
+ They make good cooks. 1 1 21
984
+ there is nothing to do. 1 1 23
985
+ John has something for Bill to do. 1 1 34
986
+ I am counting on Bill to incriminate himself. 1 1 45
987
+ On whom are you counting to incriminate himself? 1 1 48
988
+ I am counting on Bill to get there on time. 1 1 43
989
+ I would prefer to leave. 1 1 24
990
+ I would hate for John to leave. 1 1 31
991
+ I would prefer for John to leave. 1 1 33
992
+ It was hated for John to leave. 0 1 31
993
+ John decided for Bill to get the prize. 0 1 39
994
+ John decided Bill to get the prize. 0 1 35
995
+ to die is no fun. 1 1 17
996
+ John wants to leave. 1 1 20
997
+ John counted on Bill to get there on time. 1 1 42
998
+ I bought Bill a book to read. 1 1 29
999
+ John told Mary that it would be important to leave early. 1 1 57
1000
+ John told Mary that it was important to Fred to leave early. 1 1 60
1001
+ John, told Mary that it would be appropriate to leave together. 1 1 63
1002
+ the election of John president surprised me. 0 1 44
1003
+ John's arriving dead surprised me. 1 1 34
1004
+ the attempt by John to leave surprised me. 1 1 42
1005
+ John left orders to follow Pete. 1 1 32
1006
+ John left us orders to follow Pete. 1 1 35
1007
+ John left orders not to be disturbed. 1 1 37
1008
+ That he is here is clear. 1 1 25
1009
+ It is a problem that he is here. 1 1 32
1010
+ It bothers me that he is here. 1 1 30
1011
+ John regretted it that Bill had a good time. 1 1 44
1012
+ John believes it that Bill is here. 0 1 35
1013
+ John believes it sincerely that Bill is here. 0 1 45
1014
+ John is aware of it that Bill is here. 0 1 38
1015
+ John felt it that Bill was tardy. 0 1 33
1016
+ John believed it that Bill was tardy. 0 1 37
1017
+ It was believed that Bill was tardy. 1 1 36
1018
+ That John is reluctant seems. 0 1 29
1019
+ It is the problem that he is here. 0 1 34
1020
+ That he is here is the problem. 1 1 31
1021
+ The problem we are discussing is George. 1 1 40
1022
+ It is to give up to leave. 0 1 26
1023
+ It would prove our theory to be untenable for carrots to be vegetables. 1 1 71
1024
+ It was believed to be illegal by them to do that. 0 1 49
1025
+ John grudgingly accepted judgments of his incompetence as an auto mechanic. 1 1 75
1026
+ It was to John that I gave the book. 1 1 36
1027
+ I bought it to read. 1 1 20
1028
+ I bought it to give to Pete. 1 1 28
1029
+ I gave it to Pete to take to the fair. 1 1 38
1030
+ I gave Pete the book to impress. 0 1 32
1031
+ I wrote to Bill. 1 1 16
1032
+ I presented it to Bill to read. 1 1 31
1033
+ I presented Bill with it to read. 0 1 33
1034
+ I gave a book to Bill to read. 1 1 30
1035
+ John thinks it would upset himself to die. 1 1 42
1036
+ John made Bill mad at himself. 1 1 30
1037
+ John made Bill master of himself. 1 1 33
1038
+ The correspondence school made Bill a good typist. 1 1 50
1039
+ The correspondence school sent Bill a good typist. 1 1 50
1040
+ John considers Bill silly. 1 1 26
1041
+ John considers Bill to be silly. 1 1 32
1042
+ John bought a dog for himself to play with. 0 1 43
1043
+ John arranged for himself to get the prize. 1 1 43
1044
+ John talked to Bill about himself. 1 1 34
eval_cola_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "eval_cola_loss": 0.6395143270492554,
3
+ "eval_cola_matthews_correlation": -0.03589254563226399,
4
+ "eval_cola_runtime": 7.7785,
5
+ "eval_cola_samples_per_second": 134.088,
6
+ "eval_cola_steps_per_second": 16.841,
7
+ "eval_samples": 1043
8
+ }
predict_results_cola.txt ADDED
@@ -0,0 +1,1064 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ index prediction
2
+ 0 acceptable
3
+ 1 acceptable
4
+ 2 acceptable
5
+ 3 acceptable
6
+ 4 acceptable
7
+ 5 acceptable
8
+ 6 acceptable
9
+ 7 acceptable
10
+ 8 acceptable
11
+ 9 acceptable
12
+ 10 acceptable
13
+ 11 acceptable
14
+ 12 acceptable
15
+ 13 acceptable
16
+ 14 acceptable
17
+ 15 acceptable
18
+ 16 acceptable
19
+ 17 acceptable
20
+ 18 acceptable
21
+ 19 acceptable
22
+ 20 acceptable
23
+ 21 acceptable
24
+ 22 acceptable
25
+ 23 acceptable
26
+ 24 acceptable
27
+ 25 acceptable
28
+ 26 acceptable
29
+ 27 acceptable
30
+ 28 acceptable
31
+ 29 acceptable
32
+ 30 acceptable
33
+ 31 acceptable
34
+ 32 acceptable
35
+ 33 acceptable
36
+ 34 acceptable
37
+ 35 acceptable
38
+ 36 acceptable
39
+ 37 acceptable
40
+ 38 acceptable
41
+ 39 acceptable
42
+ 40 acceptable
43
+ 41 acceptable
44
+ 42 acceptable
45
+ 43 acceptable
46
+ 44 acceptable
47
+ 45 acceptable
48
+ 46 acceptable
49
+ 47 acceptable
50
+ 48 acceptable
51
+ 49 acceptable
52
+ 50 acceptable
53
+ 51 acceptable
54
+ 52 acceptable
55
+ 53 acceptable
56
+ 54 acceptable
57
+ 55 acceptable
58
+ 56 acceptable
59
+ 57 acceptable
60
+ 58 acceptable
61
+ 59 acceptable
62
+ 60 acceptable
63
+ 61 acceptable
64
+ 62 acceptable
65
+ 63 acceptable
66
+ 64 acceptable
67
+ 65 acceptable
68
+ 66 acceptable
69
+ 67 acceptable
70
+ 68 acceptable
71
+ 69 acceptable
72
+ 70 acceptable
73
+ 71 acceptable
74
+ 72 acceptable
75
+ 73 acceptable
76
+ 74 acceptable
77
+ 75 acceptable
78
+ 76 acceptable
79
+ 77 acceptable
80
+ 78 acceptable
81
+ 79 acceptable
82
+ 80 acceptable
83
+ 81 acceptable
84
+ 82 acceptable
85
+ 83 acceptable
86
+ 84 acceptable
87
+ 85 acceptable
88
+ 86 acceptable
89
+ 87 acceptable
90
+ 88 acceptable
91
+ 89 acceptable
92
+ 90 acceptable
93
+ 91 acceptable
94
+ 92 acceptable
95
+ 93 acceptable
96
+ 94 acceptable
97
+ 95 acceptable
98
+ 96 acceptable
99
+ 97 acceptable
100
+ 98 acceptable
101
+ 99 acceptable
102
+ 100 acceptable
103
+ 101 acceptable
104
+ 102 acceptable
105
+ 103 acceptable
106
+ 104 acceptable
107
+ 105 acceptable
108
+ 106 acceptable
109
+ 107 acceptable
110
+ 108 acceptable
111
+ 109 acceptable
112
+ 110 acceptable
113
+ 111 acceptable
114
+ 112 acceptable
115
+ 113 acceptable
116
+ 114 acceptable
117
+ 115 acceptable
118
+ 116 acceptable
119
+ 117 acceptable
120
+ 118 acceptable
121
+ 119 acceptable
122
+ 120 acceptable
123
+ 121 acceptable
124
+ 122 acceptable
125
+ 123 acceptable
126
+ 124 acceptable
127
+ 125 acceptable
128
+ 126 acceptable
129
+ 127 acceptable
130
+ 128 acceptable
131
+ 129 acceptable
132
+ 130 acceptable
133
+ 131 acceptable
134
+ 132 acceptable
135
+ 133 acceptable
136
+ 134 acceptable
137
+ 135 acceptable
138
+ 136 acceptable
139
+ 137 acceptable
140
+ 138 acceptable
141
+ 139 acceptable
142
+ 140 acceptable
143
+ 141 acceptable
144
+ 142 acceptable
145
+ 143 acceptable
146
+ 144 acceptable
147
+ 145 acceptable
148
+ 146 acceptable
149
+ 147 acceptable
150
+ 148 acceptable
151
+ 149 acceptable
152
+ 150 acceptable
153
+ 151 acceptable
154
+ 152 acceptable
155
+ 153 acceptable
156
+ 154 acceptable
157
+ 155 acceptable
158
+ 156 acceptable
159
+ 157 acceptable
160
+ 158 acceptable
161
+ 159 acceptable
162
+ 160 acceptable
163
+ 161 acceptable
164
+ 162 acceptable
165
+ 163 acceptable
166
+ 164 acceptable
167
+ 165 acceptable
168
+ 166 acceptable
169
+ 167 acceptable
170
+ 168 acceptable
171
+ 169 acceptable
172
+ 170 acceptable
173
+ 171 acceptable
174
+ 172 acceptable
175
+ 173 acceptable
176
+ 174 acceptable
177
+ 175 acceptable
178
+ 176 acceptable
179
+ 177 acceptable
180
+ 178 acceptable
181
+ 179 acceptable
182
+ 180 acceptable
183
+ 181 acceptable
184
+ 182 acceptable
185
+ 183 acceptable
186
+ 184 acceptable
187
+ 185 acceptable
188
+ 186 acceptable
189
+ 187 acceptable
190
+ 188 acceptable
191
+ 189 acceptable
192
+ 190 acceptable
193
+ 191 acceptable
194
+ 192 acceptable
195
+ 193 acceptable
196
+ 194 acceptable
197
+ 195 acceptable
198
+ 196 acceptable
199
+ 197 acceptable
200
+ 198 acceptable
201
+ 199 acceptable
202
+ 200 acceptable
203
+ 201 acceptable
204
+ 202 acceptable
205
+ 203 acceptable
206
+ 204 acceptable
207
+ 205 acceptable
208
+ 206 acceptable
209
+ 207 acceptable
210
+ 208 acceptable
211
+ 209 acceptable
212
+ 210 acceptable
213
+ 211 acceptable
214
+ 212 acceptable
215
+ 213 acceptable
216
+ 214 acceptable
217
+ 215 acceptable
218
+ 216 acceptable
219
+ 217 acceptable
220
+ 218 acceptable
221
+ 219 acceptable
222
+ 220 acceptable
223
+ 221 acceptable
224
+ 222 acceptable
225
+ 223 acceptable
226
+ 224 acceptable
227
+ 225 acceptable
228
+ 226 acceptable
229
+ 227 acceptable
230
+ 228 acceptable
231
+ 229 acceptable
232
+ 230 acceptable
233
+ 231 acceptable
234
+ 232 acceptable
235
+ 233 acceptable
236
+ 234 acceptable
237
+ 235 acceptable
238
+ 236 acceptable
239
+ 237 acceptable
240
+ 238 acceptable
241
+ 239 acceptable
242
+ 240 acceptable
243
+ 241 acceptable
244
+ 242 acceptable
245
+ 243 acceptable
246
+ 244 acceptable
247
+ 245 acceptable
248
+ 246 acceptable
249
+ 247 acceptable
250
+ 248 acceptable
251
+ 249 acceptable
252
+ 250 acceptable
253
+ 251 acceptable
254
+ 252 acceptable
255
+ 253 acceptable
256
+ 254 acceptable
257
+ 255 acceptable
258
+ 256 acceptable
259
+ 257 acceptable
260
+ 258 acceptable
261
+ 259 acceptable
262
+ 260 acceptable
263
+ 261 acceptable
264
+ 262 acceptable
265
+ 263 acceptable
266
+ 264 acceptable
267
+ 265 acceptable
268
+ 266 acceptable
269
+ 267 acceptable
270
+ 268 acceptable
271
+ 269 acceptable
272
+ 270 acceptable
273
+ 271 acceptable
274
+ 272 acceptable
275
+ 273 acceptable
276
+ 274 acceptable
277
+ 275 acceptable
278
+ 276 acceptable
279
+ 277 acceptable
280
+ 278 acceptable
281
+ 279 acceptable
282
+ 280 acceptable
283
+ 281 acceptable
284
+ 282 acceptable
285
+ 283 acceptable
286
+ 284 acceptable
287
+ 285 acceptable
288
+ 286 acceptable
289
+ 287 acceptable
290
+ 288 acceptable
291
+ 289 acceptable
292
+ 290 acceptable
293
+ 291 acceptable
294
+ 292 acceptable
295
+ 293 acceptable
296
+ 294 acceptable
297
+ 295 acceptable
298
+ 296 acceptable
299
+ 297 acceptable
300
+ 298 acceptable
301
+ 299 acceptable
302
+ 300 acceptable
303
+ 301 acceptable
304
+ 302 acceptable
305
+ 303 acceptable
306
+ 304 acceptable
307
+ 305 acceptable
308
+ 306 acceptable
309
+ 307 acceptable
310
+ 308 acceptable
311
+ 309 acceptable
312
+ 310 acceptable
313
+ 311 acceptable
314
+ 312 acceptable
315
+ 313 acceptable
316
+ 314 acceptable
317
+ 315 acceptable
318
+ 316 acceptable
319
+ 317 acceptable
320
+ 318 acceptable
321
+ 319 acceptable
322
+ 320 acceptable
323
+ 321 acceptable
324
+ 322 acceptable
325
+ 323 acceptable
326
+ 324 acceptable
327
+ 325 acceptable
328
+ 326 acceptable
329
+ 327 acceptable
330
+ 328 acceptable
331
+ 329 acceptable
332
+ 330 acceptable
333
+ 331 acceptable
334
+ 332 acceptable
335
+ 333 acceptable
336
+ 334 acceptable
337
+ 335 acceptable
338
+ 336 acceptable
339
+ 337 acceptable
340
+ 338 acceptable
341
+ 339 acceptable
342
+ 340 acceptable
343
+ 341 acceptable
344
+ 342 acceptable
345
+ 343 acceptable
346
+ 344 acceptable
347
+ 345 acceptable
348
+ 346 acceptable
349
+ 347 acceptable
350
+ 348 acceptable
351
+ 349 acceptable
352
+ 350 acceptable
353
+ 351 acceptable
354
+ 352 acceptable
355
+ 353 acceptable
356
+ 354 acceptable
357
+ 355 acceptable
358
+ 356 acceptable
359
+ 357 acceptable
360
+ 358 acceptable
361
+ 359 acceptable
362
+ 360 acceptable
363
+ 361 acceptable
364
+ 362 acceptable
365
+ 363 acceptable
366
+ 364 acceptable
367
+ 365 acceptable
368
+ 366 acceptable
369
+ 367 acceptable
370
+ 368 acceptable
371
+ 369 acceptable
372
+ 370 acceptable
373
+ 371 acceptable
374
+ 372 acceptable
375
+ 373 acceptable
376
+ 374 acceptable
377
+ 375 acceptable
378
+ 376 acceptable
379
+ 377 acceptable
380
+ 378 acceptable
381
+ 379 acceptable
382
+ 380 acceptable
383
+ 381 acceptable
384
+ 382 acceptable
385
+ 383 acceptable
386
+ 384 acceptable
387
+ 385 acceptable
388
+ 386 acceptable
389
+ 387 acceptable
390
+ 388 acceptable
391
+ 389 acceptable
392
+ 390 acceptable
393
+ 391 acceptable
394
+ 392 acceptable
395
+ 393 acceptable
396
+ 394 acceptable
397
+ 395 acceptable
398
+ 396 acceptable
399
+ 397 acceptable
400
+ 398 acceptable
401
+ 399 acceptable
402
+ 400 acceptable
403
+ 401 acceptable
404
+ 402 acceptable
405
+ 403 acceptable
406
+ 404 acceptable
407
+ 405 acceptable
408
+ 406 acceptable
409
+ 407 acceptable
410
+ 408 acceptable
411
+ 409 acceptable
412
+ 410 acceptable
413
+ 411 acceptable
414
+ 412 acceptable
415
+ 413 acceptable
416
+ 414 acceptable
417
+ 415 acceptable
418
+ 416 acceptable
419
+ 417 acceptable
420
+ 418 acceptable
421
+ 419 acceptable
422
+ 420 acceptable
423
+ 421 acceptable
424
+ 422 acceptable
425
+ 423 acceptable
426
+ 424 acceptable
427
+ 425 acceptable
428
+ 426 acceptable
429
+ 427 acceptable
430
+ 428 acceptable
431
+ 429 acceptable
432
+ 430 acceptable
433
+ 431 acceptable
434
+ 432 acceptable
435
+ 433 acceptable
436
+ 434 acceptable
437
+ 435 acceptable
438
+ 436 acceptable
439
+ 437 acceptable
440
+ 438 acceptable
441
+ 439 acceptable
442
+ 440 acceptable
443
+ 441 acceptable
444
+ 442 acceptable
445
+ 443 acceptable
446
+ 444 acceptable
447
+ 445 acceptable
448
+ 446 acceptable
449
+ 447 acceptable
450
+ 448 acceptable
451
+ 449 acceptable
452
+ 450 acceptable
453
+ 451 acceptable
454
+ 452 acceptable
455
+ 453 acceptable
456
+ 454 acceptable
457
+ 455 acceptable
458
+ 456 acceptable
459
+ 457 acceptable
460
+ 458 acceptable
461
+ 459 acceptable
462
+ 460 acceptable
463
+ 461 acceptable
464
+ 462 acceptable
465
+ 463 acceptable
466
+ 464 acceptable
467
+ 465 acceptable
468
+ 466 acceptable
469
+ 467 acceptable
470
+ 468 acceptable
471
+ 469 acceptable
472
+ 470 acceptable
473
+ 471 acceptable
474
+ 472 acceptable
475
+ 473 acceptable
476
+ 474 acceptable
477
+ 475 acceptable
478
+ 476 acceptable
479
+ 477 acceptable
480
+ 478 acceptable
481
+ 479 acceptable
482
+ 480 acceptable
483
+ 481 acceptable
484
+ 482 acceptable
485
+ 483 acceptable
486
+ 484 acceptable
487
+ 485 acceptable
488
+ 486 acceptable
489
+ 487 acceptable
490
+ 488 acceptable
491
+ 489 acceptable
492
+ 490 acceptable
493
+ 491 acceptable
494
+ 492 acceptable
495
+ 493 acceptable
496
+ 494 acceptable
497
+ 495 acceptable
498
+ 496 acceptable
499
+ 497 acceptable
500
+ 498 acceptable
501
+ 499 acceptable
502
+ 500 acceptable
503
+ 501 acceptable
504
+ 502 acceptable
505
+ 503 acceptable
506
+ 504 acceptable
507
+ 505 acceptable
508
+ 506 acceptable
509
+ 507 acceptable
510
+ 508 acceptable
511
+ 509 acceptable
512
+ 510 acceptable
513
+ 511 acceptable
514
+ 512 acceptable
515
+ 513 acceptable
516
+ 514 acceptable
517
+ 515 acceptable
518
+ 516 acceptable
519
+ 517 acceptable
520
+ 518 acceptable
521
+ 519 acceptable
522
+ 520 acceptable
523
+ 521 acceptable
524
+ 522 acceptable
525
+ 523 acceptable
526
+ 524 acceptable
527
+ 525 acceptable
528
+ 526 acceptable
529
+ 527 acceptable
530
+ 528 acceptable
531
+ 529 acceptable
532
+ 530 acceptable
533
+ 531 acceptable
534
+ 532 acceptable
535
+ 533 acceptable
536
+ 534 acceptable
537
+ 535 acceptable
538
+ 536 acceptable
539
+ 537 acceptable
540
+ 538 acceptable
541
+ 539 acceptable
542
+ 540 acceptable
543
+ 541 acceptable
544
+ 542 acceptable
545
+ 543 acceptable
546
+ 544 acceptable
547
+ 545 acceptable
548
+ 546 acceptable
549
+ 547 acceptable
550
+ 548 acceptable
551
+ 549 acceptable
552
+ 550 acceptable
553
+ 551 acceptable
554
+ 552 acceptable
555
+ 553 acceptable
556
+ 554 acceptable
557
+ 555 acceptable
558
+ 556 acceptable
559
+ 557 acceptable
560
+ 558 acceptable
561
+ 559 acceptable
562
+ 560 acceptable
563
+ 561 acceptable
564
+ 562 acceptable
565
+ 563 acceptable
566
+ 564 acceptable
567
+ 565 acceptable
568
+ 566 acceptable
569
+ 567 acceptable
570
+ 568 acceptable
571
+ 569 acceptable
572
+ 570 acceptable
573
+ 571 acceptable
574
+ 572 acceptable
575
+ 573 acceptable
576
+ 574 acceptable
577
+ 575 acceptable
578
+ 576 acceptable
579
+ 577 unacceptable
580
+ 578 acceptable
581
+ 579 acceptable
582
+ 580 acceptable
583
+ 581 acceptable
584
+ 582 acceptable
585
+ 583 acceptable
586
+ 584 acceptable
587
+ 585 acceptable
588
+ 586 acceptable
589
+ 587 acceptable
590
+ 588 acceptable
591
+ 589 acceptable
592
+ 590 acceptable
593
+ 591 acceptable
594
+ 592 acceptable
595
+ 593 acceptable
596
+ 594 acceptable
597
+ 595 acceptable
598
+ 596 acceptable
599
+ 597 acceptable
600
+ 598 acceptable
601
+ 599 acceptable
602
+ 600 acceptable
603
+ 601 acceptable
604
+ 602 acceptable
605
+ 603 acceptable
606
+ 604 acceptable
607
+ 605 acceptable
608
+ 606 acceptable
609
+ 607 acceptable
610
+ 608 acceptable
611
+ 609 acceptable
612
+ 610 acceptable
613
+ 611 acceptable
614
+ 612 acceptable
615
+ 613 acceptable
616
+ 614 acceptable
617
+ 615 acceptable
618
+ 616 acceptable
619
+ 617 acceptable
620
+ 618 acceptable
621
+ 619 acceptable
622
+ 620 acceptable
623
+ 621 acceptable
624
+ 622 acceptable
625
+ 623 acceptable
626
+ 624 acceptable
627
+ 625 acceptable
628
+ 626 acceptable
629
+ 627 acceptable
630
+ 628 acceptable
631
+ 629 acceptable
632
+ 630 acceptable
633
+ 631 acceptable
634
+ 632 acceptable
635
+ 633 acceptable
636
+ 634 acceptable
637
+ 635 acceptable
638
+ 636 acceptable
639
+ 637 acceptable
640
+ 638 acceptable
641
+ 639 acceptable
642
+ 640 acceptable
643
+ 641 acceptable
644
+ 642 acceptable
645
+ 643 acceptable
646
+ 644 acceptable
647
+ 645 acceptable
648
+ 646 acceptable
649
+ 647 acceptable
650
+ 648 acceptable
651
+ 649 acceptable
652
+ 650 acceptable
653
+ 651 acceptable
654
+ 652 acceptable
655
+ 653 acceptable
656
+ 654 acceptable
657
+ 655 acceptable
658
+ 656 acceptable
659
+ 657 acceptable
660
+ 658 acceptable
661
+ 659 acceptable
662
+ 660 acceptable
663
+ 661 acceptable
664
+ 662 acceptable
665
+ 663 acceptable
666
+ 664 acceptable
667
+ 665 acceptable
668
+ 666 acceptable
669
+ 667 acceptable
670
+ 668 acceptable
671
+ 669 acceptable
672
+ 670 acceptable
673
+ 671 acceptable
674
+ 672 acceptable
675
+ 673 acceptable
676
+ 674 acceptable
677
+ 675 acceptable
678
+ 676 acceptable
679
+ 677 acceptable
680
+ 678 acceptable
681
+ 679 acceptable
682
+ 680 acceptable
683
+ 681 acceptable
684
+ 682 acceptable
685
+ 683 acceptable
686
+ 684 acceptable
687
+ 685 acceptable
688
+ 686 acceptable
689
+ 687 acceptable
690
+ 688 acceptable
691
+ 689 acceptable
692
+ 690 acceptable
693
+ 691 acceptable
694
+ 692 acceptable
695
+ 693 acceptable
696
+ 694 acceptable
697
+ 695 acceptable
698
+ 696 acceptable
699
+ 697 acceptable
700
+ 698 acceptable
701
+ 699 acceptable
702
+ 700 acceptable
703
+ 701 acceptable
704
+ 702 acceptable
705
+ 703 acceptable
706
+ 704 acceptable
707
+ 705 acceptable
708
+ 706 acceptable
709
+ 707 acceptable
710
+ 708 acceptable
711
+ 709 acceptable
712
+ 710 acceptable
713
+ 711 acceptable
714
+ 712 acceptable
715
+ 713 acceptable
716
+ 714 acceptable
717
+ 715 acceptable
718
+ 716 acceptable
719
+ 717 acceptable
720
+ 718 acceptable
721
+ 719 acceptable
722
+ 720 acceptable
723
+ 721 acceptable
724
+ 722 acceptable
725
+ 723 acceptable
726
+ 724 acceptable
727
+ 725 acceptable
728
+ 726 acceptable
729
+ 727 acceptable
730
+ 728 acceptable
731
+ 729 acceptable
732
+ 730 acceptable
733
+ 731 acceptable
734
+ 732 acceptable
735
+ 733 acceptable
736
+ 734 acceptable
737
+ 735 acceptable
738
+ 736 acceptable
739
+ 737 acceptable
740
+ 738 acceptable
741
+ 739 acceptable
742
+ 740 acceptable
743
+ 741 acceptable
744
+ 742 acceptable
745
+ 743 acceptable
746
+ 744 acceptable
747
+ 745 acceptable
748
+ 746 acceptable
749
+ 747 acceptable
750
+ 748 acceptable
751
+ 749 acceptable
752
+ 750 acceptable
753
+ 751 acceptable
754
+ 752 acceptable
755
+ 753 acceptable
756
+ 754 acceptable
757
+ 755 acceptable
758
+ 756 acceptable
759
+ 757 acceptable
760
+ 758 acceptable
761
+ 759 acceptable
762
+ 760 acceptable
763
+ 761 acceptable
764
+ 762 acceptable
765
+ 763 acceptable
766
+ 764 acceptable
767
+ 765 acceptable
768
+ 766 acceptable
769
+ 767 acceptable
770
+ 768 acceptable
771
+ 769 acceptable
772
+ 770 acceptable
773
+ 771 acceptable
774
+ 772 acceptable
775
+ 773 acceptable
776
+ 774 acceptable
777
+ 775 acceptable
778
+ 776 acceptable
779
+ 777 acceptable
780
+ 778 acceptable
781
+ 779 acceptable
782
+ 780 acceptable
783
+ 781 acceptable
784
+ 782 acceptable
785
+ 783 acceptable
786
+ 784 acceptable
787
+ 785 acceptable
788
+ 786 acceptable
789
+ 787 acceptable
790
+ 788 acceptable
791
+ 789 acceptable
792
+ 790 acceptable
793
+ 791 acceptable
794
+ 792 acceptable
795
+ 793 acceptable
796
+ 794 acceptable
797
+ 795 acceptable
798
+ 796 acceptable
799
+ 797 acceptable
800
+ 798 acceptable
801
+ 799 acceptable
802
+ 800 acceptable
803
+ 801 acceptable
804
+ 802 acceptable
805
+ 803 acceptable
806
+ 804 acceptable
807
+ 805 acceptable
808
+ 806 acceptable
809
+ 807 acceptable
810
+ 808 acceptable
811
+ 809 acceptable
812
+ 810 acceptable
813
+ 811 acceptable
814
+ 812 acceptable
815
+ 813 acceptable
816
+ 814 acceptable
817
+ 815 acceptable
818
+ 816 acceptable
819
+ 817 acceptable
820
+ 818 acceptable
821
+ 819 acceptable
822
+ 820 acceptable
823
+ 821 acceptable
824
+ 822 acceptable
825
+ 823 acceptable
826
+ 824 acceptable
827
+ 825 acceptable
828
+ 826 acceptable
829
+ 827 acceptable
830
+ 828 acceptable
831
+ 829 acceptable
832
+ 830 acceptable
833
+ 831 acceptable
834
+ 832 acceptable
835
+ 833 acceptable
836
+ 834 acceptable
837
+ 835 acceptable
838
+ 836 acceptable
839
+ 837 acceptable
840
+ 838 acceptable
841
+ 839 acceptable
842
+ 840 acceptable
843
+ 841 acceptable
844
+ 842 acceptable
845
+ 843 acceptable
846
+ 844 acceptable
847
+ 845 acceptable
848
+ 846 acceptable
849
+ 847 acceptable
850
+ 848 acceptable
851
+ 849 acceptable
852
+ 850 acceptable
853
+ 851 acceptable
854
+ 852 acceptable
855
+ 853 acceptable
856
+ 854 acceptable
857
+ 855 acceptable
858
+ 856 acceptable
859
+ 857 acceptable
860
+ 858 acceptable
861
+ 859 acceptable
862
+ 860 acceptable
863
+ 861 acceptable
864
+ 862 acceptable
865
+ 863 acceptable
866
+ 864 acceptable
867
+ 865 acceptable
868
+ 866 acceptable
869
+ 867 acceptable
870
+ 868 acceptable
871
+ 869 acceptable
872
+ 870 acceptable
873
+ 871 acceptable
874
+ 872 acceptable
875
+ 873 acceptable
876
+ 874 acceptable
877
+ 875 acceptable
878
+ 876 acceptable
879
+ 877 acceptable
880
+ 878 acceptable
881
+ 879 acceptable
882
+ 880 acceptable
883
+ 881 acceptable
884
+ 882 acceptable
885
+ 883 acceptable
886
+ 884 acceptable
887
+ 885 acceptable
888
+ 886 acceptable
889
+ 887 acceptable
890
+ 888 acceptable
891
+ 889 acceptable
892
+ 890 acceptable
893
+ 891 acceptable
894
+ 892 acceptable
895
+ 893 acceptable
896
+ 894 acceptable
897
+ 895 acceptable
898
+ 896 acceptable
899
+ 897 acceptable
900
+ 898 acceptable
901
+ 899 acceptable
902
+ 900 acceptable
903
+ 901 acceptable
904
+ 902 acceptable
905
+ 903 acceptable
906
+ 904 acceptable
907
+ 905 acceptable
908
+ 906 acceptable
909
+ 907 acceptable
910
+ 908 acceptable
911
+ 909 acceptable
912
+ 910 acceptable
913
+ 911 acceptable
914
+ 912 acceptable
915
+ 913 acceptable
916
+ 914 unacceptable
917
+ 915 acceptable
918
+ 916 acceptable
919
+ 917 acceptable
920
+ 918 acceptable
921
+ 919 acceptable
922
+ 920 acceptable
923
+ 921 acceptable
924
+ 922 acceptable
925
+ 923 acceptable
926
+ 924 acceptable
927
+ 925 acceptable
928
+ 926 acceptable
929
+ 927 acceptable
930
+ 928 acceptable
931
+ 929 acceptable
932
+ 930 acceptable
933
+ 931 acceptable
934
+ 932 acceptable
935
+ 933 acceptable
936
+ 934 acceptable
937
+ 935 acceptable
938
+ 936 acceptable
939
+ 937 acceptable
940
+ 938 acceptable
941
+ 939 acceptable
942
+ 940 acceptable
943
+ 941 acceptable
944
+ 942 acceptable
945
+ 943 acceptable
946
+ 944 acceptable
947
+ 945 acceptable
948
+ 946 acceptable
949
+ 947 acceptable
950
+ 948 acceptable
951
+ 949 acceptable
952
+ 950 acceptable
953
+ 951 acceptable
954
+ 952 acceptable
955
+ 953 acceptable
956
+ 954 acceptable
957
+ 955 acceptable
958
+ 956 acceptable
959
+ 957 acceptable
960
+ 958 acceptable
961
+ 959 acceptable
962
+ 960 acceptable
963
+ 961 acceptable
964
+ 962 acceptable
965
+ 963 acceptable
966
+ 964 acceptable
967
+ 965 acceptable
968
+ 966 acceptable
969
+ 967 acceptable
970
+ 968 acceptable
971
+ 969 acceptable
972
+ 970 acceptable
973
+ 971 acceptable
974
+ 972 acceptable
975
+ 973 acceptable
976
+ 974 acceptable
977
+ 975 acceptable
978
+ 976 acceptable
979
+ 977 acceptable
980
+ 978 acceptable
981
+ 979 acceptable
982
+ 980 acceptable
983
+ 981 acceptable
984
+ 982 acceptable
985
+ 983 acceptable
986
+ 984 acceptable
987
+ 985 acceptable
988
+ 986 acceptable
989
+ 987 acceptable
990
+ 988 acceptable
991
+ 989 acceptable
992
+ 990 acceptable
993
+ 991 acceptable
994
+ 992 acceptable
995
+ 993 acceptable
996
+ 994 acceptable
997
+ 995 acceptable
998
+ 996 acceptable
999
+ 997 acceptable
1000
+ 998 acceptable
1001
+ 999 acceptable
1002
+ 1000 acceptable
1003
+ 1001 acceptable
1004
+ 1002 acceptable
1005
+ 1003 acceptable
1006
+ 1004 acceptable
1007
+ 1005 acceptable
1008
+ 1006 acceptable
1009
+ 1007 acceptable
1010
+ 1008 acceptable
1011
+ 1009 acceptable
1012
+ 1010 acceptable
1013
+ 1011 acceptable
1014
+ 1012 acceptable
1015
+ 1013 acceptable
1016
+ 1014 acceptable
1017
+ 1015 acceptable
1018
+ 1016 acceptable
1019
+ 1017 acceptable
1020
+ 1018 acceptable
1021
+ 1019 acceptable
1022
+ 1020 acceptable
1023
+ 1021 acceptable
1024
+ 1022 acceptable
1025
+ 1023 acceptable
1026
+ 1024 acceptable
1027
+ 1025 acceptable
1028
+ 1026 acceptable
1029
+ 1027 acceptable
1030
+ 1028 acceptable
1031
+ 1029 acceptable
1032
+ 1030 acceptable
1033
+ 1031 acceptable
1034
+ 1032 acceptable
1035
+ 1033 acceptable
1036
+ 1034 acceptable
1037
+ 1035 acceptable
1038
+ 1036 acceptable
1039
+ 1037 acceptable
1040
+ 1038 acceptable
1041
+ 1039 acceptable
1042
+ 1040 acceptable
1043
+ 1041 acceptable
1044
+ 1042 acceptable
1045
+ 1043 acceptable
1046
+ 1044 acceptable
1047
+ 1045 acceptable
1048
+ 1046 acceptable
1049
+ 1047 acceptable
1050
+ 1048 acceptable
1051
+ 1049 acceptable
1052
+ 1050 acceptable
1053
+ 1051 acceptable
1054
+ 1052 acceptable
1055
+ 1053 acceptable
1056
+ 1054 acceptable
1057
+ 1055 acceptable
1058
+ 1056 acceptable
1059
+ 1057 acceptable
1060
+ 1058 acceptable
1061
+ 1059 acceptable
1062
+ 1060 acceptable
1063
+ 1061 acceptable
1064
+ 1062 acceptable
train_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 33.33,
3
+ "train_loss": 0.6206500174782493,
4
+ "train_runtime": 3140.0413,
5
+ "train_samples": 8551,
6
+ "train_samples_per_second": 1222.914,
7
+ "train_steps_per_second": 4.777
8
+ }
trainer_state.json ADDED
@@ -0,0 +1,190 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 0.00286100001416597,
3
+ "best_model_checkpoint": "cola-pixel-handwritten-mean-vatrpp-256-64-4-2e-5-15000-42/checkpoint-100",
4
+ "epoch": 33.32835820895522,
5
+ "global_step": 1100,
6
+ "is_hyper_param_search": false,
7
+ "is_local_process_zero": true,
8
+ "is_world_process_zero": true,
9
+ "log_history": [
10
+ {
11
+ "epoch": 3.03,
12
+ "learning_rate": 9.800000000000001e-06,
13
+ "loss": 0.655,
14
+ "step": 100
15
+ },
16
+ {
17
+ "epoch": 3.03,
18
+ "eval_loss": 0.6341390013694763,
19
+ "eval_matthews_correlation": 0.00286100001416597,
20
+ "eval_runtime": 7.8213,
21
+ "eval_samples_per_second": 133.354,
22
+ "eval_steps_per_second": 16.749,
23
+ "step": 100
24
+ },
25
+ {
26
+ "epoch": 6.06,
27
+ "learning_rate": 1.98e-05,
28
+ "loss": 0.6174,
29
+ "step": 200
30
+ },
31
+ {
32
+ "epoch": 6.06,
33
+ "eval_loss": 0.6281591057777405,
34
+ "eval_matthews_correlation": 0.0,
35
+ "eval_runtime": 7.9503,
36
+ "eval_samples_per_second": 131.19,
37
+ "eval_steps_per_second": 16.477,
38
+ "step": 200
39
+ },
40
+ {
41
+ "epoch": 9.09,
42
+ "learning_rate": 1.9867567567567568e-05,
43
+ "loss": 0.6196,
44
+ "step": 300
45
+ },
46
+ {
47
+ "epoch": 9.09,
48
+ "eval_loss": 0.6198328137397766,
49
+ "eval_matthews_correlation": 0.0,
50
+ "eval_runtime": 7.8212,
51
+ "eval_samples_per_second": 133.356,
52
+ "eval_steps_per_second": 16.749,
53
+ "step": 300
54
+ },
55
+ {
56
+ "epoch": 12.12,
57
+ "learning_rate": 1.9733783783783785e-05,
58
+ "loss": 0.6158,
59
+ "step": 400
60
+ },
61
+ {
62
+ "epoch": 12.12,
63
+ "eval_loss": 0.6199322938919067,
64
+ "eval_matthews_correlation": 0.0,
65
+ "eval_runtime": 7.8954,
66
+ "eval_samples_per_second": 132.102,
67
+ "eval_steps_per_second": 16.592,
68
+ "step": 400
69
+ },
70
+ {
71
+ "epoch": 15.15,
72
+ "learning_rate": 1.9600000000000002e-05,
73
+ "loss": 0.6175,
74
+ "step": 500
75
+ },
76
+ {
77
+ "epoch": 15.15,
78
+ "eval_loss": 0.6180645823478699,
79
+ "eval_matthews_correlation": 0.0,
80
+ "eval_runtime": 8.2397,
81
+ "eval_samples_per_second": 126.583,
82
+ "eval_steps_per_second": 15.899,
83
+ "step": 500
84
+ },
85
+ {
86
+ "epoch": 18.18,
87
+ "learning_rate": 1.9464864864864867e-05,
88
+ "loss": 0.6152,
89
+ "step": 600
90
+ },
91
+ {
92
+ "epoch": 18.18,
93
+ "eval_loss": 0.619079053401947,
94
+ "eval_matthews_correlation": 0.0,
95
+ "eval_runtime": 7.8304,
96
+ "eval_samples_per_second": 133.198,
97
+ "eval_steps_per_second": 16.73,
98
+ "step": 600
99
+ },
100
+ {
101
+ "epoch": 21.21,
102
+ "learning_rate": 1.932972972972973e-05,
103
+ "loss": 0.617,
104
+ "step": 700
105
+ },
106
+ {
107
+ "epoch": 21.21,
108
+ "eval_loss": 0.6184842586517334,
109
+ "eval_matthews_correlation": 0.0,
110
+ "eval_runtime": 8.2213,
111
+ "eval_samples_per_second": 126.865,
112
+ "eval_steps_per_second": 15.934,
113
+ "step": 700
114
+ },
115
+ {
116
+ "epoch": 24.24,
117
+ "learning_rate": 1.9194594594594596e-05,
118
+ "loss": 0.6191,
119
+ "step": 800
120
+ },
121
+ {
122
+ "epoch": 24.24,
123
+ "eval_loss": 0.6185177564620972,
124
+ "eval_matthews_correlation": 0.0,
125
+ "eval_runtime": 7.887,
126
+ "eval_samples_per_second": 132.243,
127
+ "eval_steps_per_second": 16.61,
128
+ "step": 800
129
+ },
130
+ {
131
+ "epoch": 27.27,
132
+ "learning_rate": 1.905945945945946e-05,
133
+ "loss": 0.6162,
134
+ "step": 900
135
+ },
136
+ {
137
+ "epoch": 27.27,
138
+ "eval_loss": 0.6183082461357117,
139
+ "eval_matthews_correlation": 0.0,
140
+ "eval_runtime": 7.8268,
141
+ "eval_samples_per_second": 133.26,
142
+ "eval_steps_per_second": 16.737,
143
+ "step": 900
144
+ },
145
+ {
146
+ "epoch": 30.3,
147
+ "learning_rate": 1.8924324324324325e-05,
148
+ "loss": 0.6166,
149
+ "step": 1000
150
+ },
151
+ {
152
+ "epoch": 30.3,
153
+ "eval_loss": 0.6183302402496338,
154
+ "eval_matthews_correlation": 0.0,
155
+ "eval_runtime": 7.795,
156
+ "eval_samples_per_second": 133.803,
157
+ "eval_steps_per_second": 16.806,
158
+ "step": 1000
159
+ },
160
+ {
161
+ "epoch": 33.33,
162
+ "learning_rate": 1.878918918918919e-05,
163
+ "loss": 0.6177,
164
+ "step": 1100
165
+ },
166
+ {
167
+ "epoch": 33.33,
168
+ "eval_loss": 0.6182125210762024,
169
+ "eval_matthews_correlation": 0.0,
170
+ "eval_runtime": 7.8695,
171
+ "eval_samples_per_second": 132.537,
172
+ "eval_steps_per_second": 16.647,
173
+ "step": 1100
174
+ },
175
+ {
176
+ "epoch": 33.33,
177
+ "step": 1100,
178
+ "total_flos": 7.341930418964005e+18,
179
+ "train_loss": 0.6206500174782493,
180
+ "train_runtime": 3140.0413,
181
+ "train_samples_per_second": 1222.914,
182
+ "train_steps_per_second": 4.777
183
+ }
184
+ ],
185
+ "max_steps": 15000,
186
+ "num_train_epochs": 455,
187
+ "total_flos": 7.341930418964005e+18,
188
+ "trial_name": null,
189
+ "trial_params": null
190
+ }