| Results after fine-tuning distilbert in 80% of 15189 instances | |
| **(Model still under development)** | |
| 20it [00:11, 1.85it/s]Train: wpb=2121, num_updates=20, accuracy=44.1, loss=0.00\ | |
| 50it [00:28, 1.76it/s]Train: wpb=2121, num_updates=50, accuracy=55.4, loss=0.00\ | |
| 100it [00:55, 1.88it/s]Train: wpb=2117, num_updates=100, accuracy=64.5, loss=0.00\ | |
| 200it [01:48, 1.85it/s]Train: wpb=2132, num_updates=200, accuracy=71.6, loss=0.00\ | |
| 300it [02:42, 1.88it/s]Train: wpb=2147, num_updates=300, accuracy=75.1, loss=0.00\ | |
| 380it [03:24, 1.86it/s]\ | |
| Train: wpb=2142, num_updates=380, accuracy=76.9, loss=0.00\ | |
| | epoch 000 | train accuracy=76.9%, train loss=0.00\ | |
| | epoch 000 | valid accuracy=85.7%, valid loss=0.00\ | |
| 20it [00:10, 1.85it/s]Train: wpb=2121, num_updates=20, accuracy=84.6, loss=0.00\ | |
| 50it [00:27, 1.77it/s]Train: wpb=2121, num_updates=50, accuracy=84.6, loss=0.00\ | |
| 100it [00:54, 1.87it/s]Train: wpb=2117, num_updates=100, accuracy=85.1, loss=0.00\ | |
| 200it [01:47, 1.86it/s]Train: wpb=2132, num_updates=200, accuracy=85.4, loss=0.00\ | |
| 300it [02:41, 1.88it/s]Train: wpb=2147, num_updates=300, accuracy=85.6, loss=0.00\ | |
| 380it [03:24, 1.86it/s]\ | |
| Train: wpb=2142, num_updates=380, accuracy=85.8, loss=0.00\ | |
| | epoch 001 | train accuracy=85.8%, train loss=0.00\ | |
| | epoch 001 | valid accuracy=88.3%, valid loss=0.00 | |
| 20it [00:10, 1.86it/s]Train: wpb=2121, num_updates=20, accuracy=87.3, loss=0.00\ | |
| 50it [00:27, 1.77it/s]Train: wpb=2121, num_updates=50, accuracy=87.0, loss=0.00\ | |
| 100it [00:54, 1.88it/s]Train: wpb=2117, num_updates=100, accuracy=87.2, loss=0.00\ | |
| 200it [01:47, 1.85it/s]Train: wpb=2132, num_updates=200, accuracy=87.2, loss=0.00\ | |
| 300it [02:41, 1.88it/s]Train: wpb=2147, num_updates=300, accuracy=87.2, loss=0.00\ | |
| 380it [03:23, 1.86it/s]\ | |
| Train: wpb=2142, num_updates=380, accuracy=87.3, loss=0.00\ | |
| | epoch 002 | train accuracy=87.3%, train loss=0.00\ | |
| | epoch 002 | valid accuracy=89.3%, valid loss=0.00 | |
| We have to change the loss function... It seems to be a problem... | |
| **You can evaluate the performance of our model by writing the following example:** | |
| *"google chrome before 18. 0. 1025. 142 does not properly validate the renderer's navigation requests, which has unspecified impact and remote attack vectors."* | |
| The result, for each token, should be similar : | |
| ['B-vendor', 'B-application', 'B-version', 'I-version', 'I-version', 'I-version', 'I-version', 'I-version', 'I-version', 'I-version', 'I-version', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B-relevant_term', 'O', 'O', 'O', 'O', 'O', 'O', 'B-relevant_term', 'B-relevant_term', 'O', 'O'] | |
| Different possible classes that are detected: | |
| ['I-update', 'I-version', 'B-programming language', 'B-relevant_term', 'B-parameter', 'I-relevant_term', 'B-vendor', 'B-function', 'B-version', 'B-hardware', 'I-application', 'B-os', 'O', 'B-cve id', 'B-update', 'I-edition', 'I-hardware', 'I-os', 'B-edition', 'B-application', 'B-language', 'B-file', 'B-method', 'I-vendor'] | |