aapot
commited on
Commit
·
45b39ff
1
Parent(s):
53e64e0
Update README
Browse files
README.md
CHANGED
@@ -79,7 +79,7 @@ The cross-encoders that we use to determine similarity of texts are also trained
|
|
79 |
|
80 |
For example, it's possible that the models have encoded a social bias that certain ethnicities are more often involved in violent situations. If this were the case, it is possible that videos about people of one ethnicity may be more likely to be rated similar to videos about violent situations. This could be evaluated by applying the model to synthetic video pairs crafted to test these situations. There is also [active research](https://www.aaai.org/AAAI22Papers/AISI-7742.KanekoM.pdf) in measuring bias in language models, as part of the broader field of [AI fairness](https://facctconference.org/2022/index.html).
|
81 |
|
82 |
-
|
83 |
|
84 |
A more difficult issue is the multilingual nature of our data. For the pretrained cross-encoders in our model, we used the [mmarco-mMiniLMv2-L12-H384-v1](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1) model which supports a set of 100 languages (the original mMiniLMv2 base model) including English, German, Spanish and Chinese. However, it is reasonable to expect that the model's performance varies among the languages that it supports. The impact can vary — the model may fail either with false positives, in which it thinks a dissimilar pair is similar, or false negatives, in which it thinks a similar pair is dissimilar. We performed a basic analysis to evaluate the performance of our model in different languages and it suggested that our model performs well across languages, but the potential differences in the quality of our labels between languages reduced our confidence.
|
85 |
|
|
|
79 |
|
80 |
For example, it's possible that the models have encoded a social bias that certain ethnicities are more often involved in violent situations. If this were the case, it is possible that videos about people of one ethnicity may be more likely to be rated similar to videos about violent situations. This could be evaluated by applying the model to synthetic video pairs crafted to test these situations. There is also [active research](https://www.aaai.org/AAAI22Papers/AISI-7742.KanekoM.pdf) in measuring bias in language models, as part of the broader field of [AI fairness](https://facctconference.org/2022/index.html).
|
81 |
|
82 |
+
We have not analyzed the biases in our model as, for our original application, potential for harm was extremely low. Care should be taken in future applications.
|
83 |
|
84 |
A more difficult issue is the multilingual nature of our data. For the pretrained cross-encoders in our model, we used the [mmarco-mMiniLMv2-L12-H384-v1](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1) model which supports a set of 100 languages (the original mMiniLMv2 base model) including English, German, Spanish and Chinese. However, it is reasonable to expect that the model's performance varies among the languages that it supports. The impact can vary — the model may fail either with false positives, in which it thinks a dissimilar pair is similar, or false negatives, in which it thinks a similar pair is dissimilar. We performed a basic analysis to evaluate the performance of our model in different languages and it suggested that our model performs well across languages, but the potential differences in the quality of our labels between languages reduced our confidence.
|
85 |
|