updated readme with training data/type info
Browse files
README.md
CHANGED
@@ -57,11 +57,11 @@ can be around 0.5 to 1 GiB depending on the used GPU.
|
|
57 |
- Base language model: [English BERT-Small](https://huggingface.co/google/bert_uncased_L-4_H-512_A-8)
|
58 |
- Insensitive to casing and accents
|
59 |
- Output dimensions: 256 (reduced with an additional dense layer)
|
60 |
-
- Training procedure:
|
61 |
|
62 |
### Training Data
|
63 |
|
64 |
-
|
65 |
|
66 |
### Evaluation Metrics
|
67 |
|
|
|
57 |
- Base language model: [English BERT-Small](https://huggingface.co/google/bert_uncased_L-4_H-512_A-8)
|
58 |
- Insensitive to casing and accents
|
59 |
- Output dimensions: 256 (reduced with an additional dense layer)
|
60 |
+
- Training procedure: A first model was trained with query-passage pairs, using the in-batch negative strategy with [this loss](https://www.sbert.net/docs/package_reference/losses.html#multiplenegativesrankingloss). A second model was then trained on query-passage-negative triplets with negatives mined from the previous model, like a variant of [ANCE](https://arxiv.org/pdf/2007.00808.pdf) but with different hyper parameters.
|
61 |
|
62 |
### Training Data
|
63 |
|
64 |
+
The model was trained on a Sinequa curated version of Google's [Natural Questions](https://ai.google.com/research/NaturalQuestions).
|
65 |
|
66 |
### Evaluation Metrics
|
67 |
|