File size: 20,923 Bytes
56bed65
 
 
28726a3
56bed65
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
---
tags:
- generated_from_trainer
base_model: microsoft/layoutlm-base-uncased
model-index:
- name: layoutlm-doclaynet
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# layoutlm-doclaynet

This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0004
- Footer: {'precision': 0.9992401215805471, 'recall': 0.9984813971146546, 'f1': 0.9988606152677554, 'number': 1317}
- Header: {'precision': 0.9988290398126464, 'recall': 0.9976608187134502, 'f1': 0.9982445874780573, 'number': 855}
- Able: {'precision': 0.9987046632124352, 'recall': 1.0, 'f1': 0.9993519118600129, 'number': 771}
- Aption: {'precision': 0.99375, 'recall': 0.9921996879875195, 'f1': 0.9929742388758782, 'number': 641}
- Ext: {'precision': 0.9927302100161551, 'recall': 0.9935327405012127, 'f1': 0.9931313131313131, 'number': 2474}
- Icture: {'precision': 0.9975216852540273, 'recall': 0.9975216852540273, 'f1': 0.9975216852540273, 'number': 807}
- Itle: {'precision': 0.9568965517241379, 'recall': 0.9736842105263158, 'f1': 0.9652173913043478, 'number': 114}
- Ootnote: {'precision': 1.0, 'recall': 0.9824561403508771, 'f1': 0.9911504424778761, 'number': 57}
- Ormula: {'precision': 0.9819819819819819, 'recall': 0.990909090909091, 'f1': 0.9864253393665158, 'number': 110}
- Overall Precision: 0.9952
- Overall Recall: 0.9955
- Overall F1: 0.9954
- Overall Accuracy: 0.9999

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15

### Training results

| Training Loss | Epoch | Step | Validation Loss | Footer                                                                                                    | Header                                                                                                   | Able                                                                                                      | Aption                                                                                                      | Ext                                                                                                         | Icture                                                                                                    | Itle                                                                                                        | Ootnote                                                                                                    | Ormula                                                                                                     | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| 0.4275        | 1.0   | 307  | 0.2122          | {'precision': 0.6709050112191474, 'recall': 0.6810933940774487, 'f1': 0.6759608138658629, 'number': 1317} | {'precision': 0.7034482758620689, 'recall': 0.7157894736842105, 'f1': 0.7095652173913042, 'number': 855} | {'precision': 0.43611705924339755, 'recall': 0.7924773022049286, 'f1': 0.5626151012891344, 'number': 771} | {'precision': 0.28087649402390436, 'recall': 0.43993759750390016, 'f1': 0.34285714285714286, 'number': 641} | {'precision': 0.33685839566675263, 'recall': 0.5278900565885206, 'f1': 0.41127381514722083, 'number': 2474} | {'precision': 0.49547920433996384, 'recall': 0.6790582403965304, 'f1': 0.5729221118661788, 'number': 807} | {'precision': 0.09734513274336283, 'recall': 0.09649122807017543, 'f1': 0.09691629955947136, 'number': 114} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 57}                                                 | {'precision': 0.10673234811165845, 'recall': 0.5909090909090909, 'f1': 0.1808066759388039, 'number': 110}  | 0.4199            | 0.6062         | 0.4961     | 0.9351           |
| 0.2047        | 2.0   | 614  | 0.1510          | {'precision': 0.8542568542568543, 'recall': 0.8990129081245254, 'f1': 0.8760636330003699, 'number': 1317} | {'precision': 0.8981132075471698, 'recall': 0.8350877192982457, 'f1': 0.8654545454545455, 'number': 855} | {'precision': 0.6210418794688458, 'recall': 0.788586251621271, 'f1': 0.694857142857143, 'number': 771}    | {'precision': 0.37422771403353927, 'recall': 0.6614664586583463, 'f1': 0.47801578354002255, 'number': 641}  | {'precision': 0.43380583043594545, 'recall': 0.6556184316895716, 'f1': 0.5221310156124256, 'number': 2474}  | {'precision': 0.46895544192841493, 'recall': 0.7955390334572491, 'f1': 0.5900735294117647, 'number': 807} | {'precision': 0.22818791946308725, 'recall': 0.2982456140350877, 'f1': 0.2585551330798479, 'number': 114}   | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 57}                                                 | {'precision': 0.11032531824611033, 'recall': 0.7090909090909091, 'f1': 0.19094247246022034, 'number': 110} | 0.5173            | 0.7425         | 0.6098     | 0.9506           |
| 0.1157        | 3.0   | 921  | 0.0578          | {'precision': 0.9427710843373494, 'recall': 0.9506454062262718, 'f1': 0.9466918714555767, 'number': 1317} | {'precision': 0.9485294117647058, 'recall': 0.9052631578947369, 'f1': 0.9263913824057451, 'number': 855} | {'precision': 0.7569296375266524, 'recall': 0.920881971465629, 'f1': 0.8308952603861907, 'number': 771}   | {'precision': 0.6173913043478261, 'recall': 0.7753510140405616, 'f1': 0.6874135546334715, 'number': 641}    | {'precision': 0.679323438039351, 'recall': 0.7954729183508489, 'f1': 0.732824427480916, 'number': 2474}     | {'precision': 0.812039312039312, 'recall': 0.8190830235439901, 'f1': 0.8155459592843924, 'number': 807}   | {'precision': 0.3493150684931507, 'recall': 0.4473684210526316, 'f1': 0.3923076923076923, 'number': 114}    | {'precision': 0.125, 'recall': 0.14035087719298245, 'f1': 0.1322314049586777, 'number': 57}                | {'precision': 0.4380952380952381, 'recall': 0.8363636363636363, 'f1': 0.5750000000000001, 'number': 110}   | 0.7499            | 0.8414         | 0.7931     | 0.9845           |
| 0.0672        | 4.0   | 1228 | 0.0574          | {'precision': 0.9497413155949741, 'recall': 0.9757023538344722, 'f1': 0.9625468164794007, 'number': 1317} | {'precision': 0.96, 'recall': 0.9543859649122807, 'f1': 0.9571847507331378, 'number': 855}               | {'precision': 0.8123543123543123, 'recall': 0.9040207522697795, 'f1': 0.8557397176181706, 'number': 771}  | {'precision': 0.5407035175879397, 'recall': 0.8393135725429017, 'f1': 0.6577017114914424, 'number': 641}    | {'precision': 0.6827350427350427, 'recall': 0.807194826192401, 'f1': 0.7397666234487869, 'number': 2474}    | {'precision': 0.6983511154219205, 'recall': 0.8921933085501859, 'f1': 0.7834602829162134, 'number': 807}  | {'precision': 0.5446428571428571, 'recall': 0.5350877192982456, 'f1': 0.5398230088495576, 'number': 114}    | {'precision': 0.25675675675675674, 'recall': 0.3333333333333333, 'f1': 0.2900763358778626, 'number': 57}   | {'precision': 0.6289308176100629, 'recall': 0.9090909090909091, 'f1': 0.7434944237918215, 'number': 110}   | 0.7458            | 0.8722         | 0.8041     | 0.9830           |
| 0.0379        | 5.0   | 1535 | 0.0258          | {'precision': 0.9848369977255497, 'recall': 0.9863325740318907, 'f1': 0.9855842185128985, 'number': 1317} | {'precision': 0.9811764705882353, 'recall': 0.9754385964912281, 'f1': 0.9782991202346041, 'number': 855} | {'precision': 0.9011264080100125, 'recall': 0.933852140077821, 'f1': 0.9171974522292994, 'number': 771}   | {'precision': 0.7589403973509934, 'recall': 0.8939157566302652, 'f1': 0.820916905444126, 'number': 641}     | {'precision': 0.8442001516300227, 'recall': 0.9001616814874697, 'f1': 0.871283255086072, 'number': 2474}    | {'precision': 0.8192771084337349, 'recall': 0.9268897149938042, 'f1': 0.8697674418604652, 'number': 807}  | {'precision': 0.6782608695652174, 'recall': 0.6842105263157895, 'f1': 0.6812227074235808, 'number': 114}    | {'precision': 0.574468085106383, 'recall': 0.47368421052631576, 'f1': 0.5192307692307692, 'number': 57}    | {'precision': 0.9464285714285714, 'recall': 0.9636363636363636, 'f1': 0.9549549549549549, 'number': 110}   | 0.8760            | 0.9253         | 0.9000     | 0.9921           |
| 0.0256        | 6.0   | 1842 | 0.0111          | {'precision': 0.9811746987951807, 'recall': 0.9893697798025817, 'f1': 0.9852551984877127, 'number': 1317} | {'precision': 0.9893992932862191, 'recall': 0.9824561403508771, 'f1': 0.9859154929577465, 'number': 855} | {'precision': 0.9606099110546379, 'recall': 0.980544747081712, 'f1': 0.9704749679075738, 'number': 771}   | {'precision': 0.8571428571428571, 'recall': 0.9173166926677067, 'f1': 0.8862094951017331, 'number': 641}    | {'precision': 0.8938223938223938, 'recall': 0.9357316087308003, 'f1': 0.9142969984202212, 'number': 2474}   | {'precision': 0.9262899262899262, 'recall': 0.9343246592317225, 'f1': 0.9302899444787168, 'number': 807}  | {'precision': 0.7666666666666667, 'recall': 0.8070175438596491, 'f1': 0.7863247863247863, 'number': 114}    | {'precision': 0.41935483870967744, 'recall': 0.45614035087719296, 'f1': 0.43697478991596644, 'number': 57} | {'precision': 0.9473684210526315, 'recall': 0.9818181818181818, 'f1': 0.9642857142857142, 'number': 110}   | 0.9227            | 0.9491         | 0.9357     | 0.9969           |
| 0.0134        | 7.0   | 2149 | 0.0093          | {'precision': 0.9879245283018868, 'recall': 0.9939255884586181, 'f1': 0.9909159727479183, 'number': 1317} | {'precision': 0.9655963302752294, 'recall': 0.9847953216374269, 'f1': 0.97510133178923, 'number': 855}   | {'precision': 0.9569620253164557, 'recall': 0.980544747081712, 'f1': 0.968609865470852, 'number': 771}    | {'precision': 0.8998505231689088, 'recall': 0.9391575663026521, 'f1': 0.9190839694656487, 'number': 641}    | {'precision': 0.9224103977944073, 'recall': 0.946645109135004, 'f1': 0.9343706363455017, 'number': 2474}    | {'precision': 0.9261904761904762, 'recall': 0.9640644361833953, 'f1': 0.9447480267152397, 'number': 807}  | {'precision': 0.7016129032258065, 'recall': 0.7631578947368421, 'f1': 0.73109243697479, 'number': 114}      | {'precision': 0.7169811320754716, 'recall': 0.6666666666666666, 'f1': 0.6909090909090909, 'number': 57}    | {'precision': 0.9818181818181818, 'recall': 0.9818181818181818, 'f1': 0.9818181818181818, 'number': 110}   | 0.9372            | 0.9603         | 0.9486     | 0.9976           |
| 0.0096        | 8.0   | 2456 | 0.0056          | {'precision': 0.9946848899012908, 'recall': 0.9946848899012908, 'f1': 0.9946848899012908, 'number': 1317} | {'precision': 0.9964830011723329, 'recall': 0.9941520467836257, 'f1': 0.9953161592505855, 'number': 855} | {'precision': 0.9871134020618557, 'recall': 0.993514915693904, 'f1': 0.9903038138332256, 'number': 771}   | {'precision': 0.9357798165137615, 'recall': 0.9547581903276131, 'f1': 0.9451737451737452, 'number': 641}    | {'precision': 0.9389131297104324, 'recall': 0.9567502021018593, 'f1': 0.9477477477477478, 'number': 2474}   | {'precision': 0.9541062801932367, 'recall': 0.9789343246592317, 'f1': 0.9663608562691132, 'number': 807}  | {'precision': 0.8032786885245902, 'recall': 0.8596491228070176, 'f1': 0.8305084745762712, 'number': 114}    | {'precision': 0.620253164556962, 'recall': 0.8596491228070176, 'f1': 0.7205882352941176, 'number': 57}     | {'precision': 0.956140350877193, 'recall': 0.990909090909091, 'f1': 0.9732142857142858, 'number': 110}     | 0.9569            | 0.9727         | 0.9647     | 0.9986           |
| 0.0052        | 9.0   | 2763 | 0.0039          | {'precision': 0.9931766489764974, 'recall': 0.9946848899012908, 'f1': 0.9939301972685887, 'number': 1317} | {'precision': 0.9976608187134502, 'recall': 0.9976608187134502, 'f1': 0.9976608187134502, 'number': 855} | {'precision': 0.9935233160621761, 'recall': 0.9948119325551232, 'f1': 0.9941672067401167, 'number': 771}  | {'precision': 0.9493087557603687, 'recall': 0.9641185647425897, 'f1': 0.956656346749226, 'number': 641}     | {'precision': 0.9686872741870735, 'recall': 0.9753435731608731, 'f1': 0.9720040281973816, 'number': 2474}   | {'precision': 0.9633699633699634, 'recall': 0.9776951672862454, 'f1': 0.970479704797048, 'number': 807}   | {'precision': 0.847457627118644, 'recall': 0.8771929824561403, 'f1': 0.8620689655172413, 'number': 114}     | {'precision': 0.9285714285714286, 'recall': 0.9122807017543859, 'f1': 0.9203539823008849, 'number': 57}    | {'precision': 0.990909090909091, 'recall': 0.990909090909091, 'f1': 0.990909090909091, 'number': 110}      | 0.9750            | 0.9811         | 0.9780     | 0.9990           |
| 0.0042        | 10.0  | 3070 | 0.0043          | {'precision': 0.9977238239757208, 'recall': 0.9984813971146546, 'f1': 0.9981024667931689, 'number': 1317} | {'precision': 0.9976553341148886, 'recall': 0.9953216374269006, 'f1': 0.996487119437939, 'number': 855}  | {'precision': 0.982051282051282, 'recall': 0.993514915693904, 'f1': 0.9877498388136685, 'number': 771}    | {'precision': 0.9521604938271605, 'recall': 0.9625585023400937, 'f1': 0.9573312645461599, 'number': 641}    | {'precision': 0.9668265387689848, 'recall': 0.9777687954729184, 'f1': 0.972266881028939, 'number': 2474}    | {'precision': 0.9672727272727273, 'recall': 0.9888475836431226, 'f1': 0.9779411764705883, 'number': 807}  | {'precision': 0.8760330578512396, 'recall': 0.9298245614035088, 'f1': 0.902127659574468, 'number': 114}     | {'precision': 0.8412698412698413, 'recall': 0.9298245614035088, 'f1': 0.8833333333333334, 'number': 57}    | {'precision': 0.990909090909091, 'recall': 0.990909090909091, 'f1': 0.990909090909091, 'number': 110}      | 0.9742            | 0.9843         | 0.9793     | 0.9988           |
| 0.0023        | 11.0  | 3377 | 0.0017          | {'precision': 0.9977220956719818, 'recall': 0.9977220956719818, 'f1': 0.9977220956719818, 'number': 1317} | {'precision': 0.9988304093567252, 'recall': 0.9988304093567252, 'f1': 0.9988304093567252, 'number': 855} | {'precision': 0.9871630295250321, 'recall': 0.9974059662775616, 'f1': 0.9922580645161291, 'number': 771}  | {'precision': 0.9720930232558139, 'recall': 0.9781591263650546, 'f1': 0.9751166407465008, 'number': 641}    | {'precision': 0.9819277108433735, 'recall': 0.9882780921584479, 'f1': 0.9850926672038678, 'number': 2474}   | {'precision': 0.9889025893958077, 'recall': 0.9938042131350682, 'f1': 0.9913473423980222, 'number': 807}  | {'precision': 0.9391304347826087, 'recall': 0.9473684210526315, 'f1': 0.9432314410480349, 'number': 114}    | {'precision': 0.9642857142857143, 'recall': 0.9473684210526315, 'f1': 0.9557522123893805, 'number': 57}    | {'precision': 0.9818181818181818, 'recall': 0.9818181818181818, 'f1': 0.9818181818181818, 'number': 110}   | 0.9865            | 0.9909         | 0.9887     | 0.9995           |
| 0.0019        | 12.0  | 3684 | 0.0011          | {'precision': 0.9992395437262357, 'recall': 0.9977220956719818, 'f1': 0.9984802431610942, 'number': 1317} | {'precision': 0.9988304093567252, 'recall': 0.9988304093567252, 'f1': 0.9988304093567252, 'number': 855} | {'precision': 0.9871630295250321, 'recall': 0.9974059662775616, 'f1': 0.9922580645161291, 'number': 771}  | {'precision': 0.9875195007800313, 'recall': 0.9875195007800313, 'f1': 0.9875195007800313, 'number': 641}    | {'precision': 0.9847205468435867, 'recall': 0.9898949070331448, 'f1': 0.9873009473896394, 'number': 2474}   | {'precision': 0.9889434889434889, 'recall': 0.9975216852540273, 'f1': 0.9932140653917335, 'number': 807}  | {'precision': 0.9487179487179487, 'recall': 0.9736842105263158, 'f1': 0.9610389610389611, 'number': 114}    | {'precision': 0.9821428571428571, 'recall': 0.9649122807017544, 'f1': 0.9734513274336283, 'number': 57}    | {'precision': 0.9316239316239316, 'recall': 0.990909090909091, 'f1': 0.960352422907489, 'number': 110}     | 0.9886            | 0.9934         | 0.9910     | 0.9996           |
| 0.0012        | 13.0  | 3991 | 0.0008          | {'precision': 0.9992401215805471, 'recall': 0.9984813971146546, 'f1': 0.9988606152677554, 'number': 1317} | {'precision': 0.9988290398126464, 'recall': 0.9976608187134502, 'f1': 0.9982445874780573, 'number': 855} | {'precision': 0.9974093264248705, 'recall': 0.9987029831387808, 'f1': 0.9980557355800389, 'number': 771}  | {'precision': 0.9875583203732504, 'recall': 0.9906396255850234, 'f1': 0.9890965732087228, 'number': 641}    | {'precision': 0.9871071716357775, 'recall': 0.9902991107518189, 'f1': 0.9887005649717514, 'number': 2474}   | {'precision': 0.9938271604938271, 'recall': 0.9975216852540273, 'f1': 0.9956709956709957, 'number': 807}  | {'precision': 0.9823008849557522, 'recall': 0.9736842105263158, 'f1': 0.9779735682819383, 'number': 114}    | {'precision': 0.9818181818181818, 'recall': 0.9473684210526315, 'f1': 0.9642857142857142, 'number': 57}    | {'precision': 0.9646017699115044, 'recall': 0.990909090909091, 'f1': 0.9775784753363229, 'number': 110}    | 0.9922            | 0.9938         | 0.9930     | 0.9998           |
| 0.0011        | 14.0  | 4298 | 0.0006          | {'precision': 0.9992406985573272, 'recall': 0.9992406985573272, 'f1': 0.9992406985573272, 'number': 1317} | {'precision': 0.9988290398126464, 'recall': 0.9976608187134502, 'f1': 0.9982445874780573, 'number': 855} | {'precision': 0.9974126778783958, 'recall': 1.0, 'f1': 0.9987046632124352, 'number': 771}                 | {'precision': 0.9890965732087228, 'recall': 0.9906396255850234, 'f1': 0.9898674980514419, 'number': 641}    | {'precision': 0.9879178413209827, 'recall': 0.9915117219078415, 'f1': 0.98971151906395, 'number': 2474}     | {'precision': 0.9975216852540273, 'recall': 0.9975216852540273, 'f1': 0.9975216852540273, 'number': 807}  | {'precision': 0.940677966101695, 'recall': 0.9736842105263158, 'f1': 0.956896551724138, 'number': 114}      | {'precision': 0.9642857142857143, 'recall': 0.9473684210526315, 'f1': 0.9557522123893805, 'number': 57}    | {'precision': 0.9732142857142857, 'recall': 0.990909090909091, 'f1': 0.9819819819819819, 'number': 110}    | 0.9923            | 0.9945         | 0.9934     | 0.9998           |
| 0.0008        | 15.0  | 4605 | 0.0004          | {'precision': 0.9992401215805471, 'recall': 0.9984813971146546, 'f1': 0.9988606152677554, 'number': 1317} | {'precision': 0.9988290398126464, 'recall': 0.9976608187134502, 'f1': 0.9982445874780573, 'number': 855} | {'precision': 0.9987046632124352, 'recall': 1.0, 'f1': 0.9993519118600129, 'number': 771}                 | {'precision': 0.99375, 'recall': 0.9921996879875195, 'f1': 0.9929742388758782, 'number': 641}               | {'precision': 0.9927302100161551, 'recall': 0.9935327405012127, 'f1': 0.9931313131313131, 'number': 2474}   | {'precision': 0.9975216852540273, 'recall': 0.9975216852540273, 'f1': 0.9975216852540273, 'number': 807}  | {'precision': 0.9568965517241379, 'recall': 0.9736842105263158, 'f1': 0.9652173913043478, 'number': 114}    | {'precision': 1.0, 'recall': 0.9824561403508771, 'f1': 0.9911504424778761, 'number': 57}                   | {'precision': 0.9819819819819819, 'recall': 0.990909090909091, 'f1': 0.9864253393665158, 'number': 110}    | 0.9952            | 0.9955         | 0.9954     | 0.9999           |


### Framework versions

- Transformers 4.26.0
- Pytorch 1.12.1
- Datasets 2.9.0
- Tokenizers 0.13.2