Change BOS token from 0 to 2 as BOS token is equal to EOS for OPT. See: https://github.com/huggingface/transformers/issues/17431 (#1)

- Change BOS token from 0 to 2 as BOS token is equal to EOS for OPT. See: https://github.com/huggingface/transformers/issues/17431 (57bc06d0f4b83079b66f1faf67f29b0157447764)


Co-authored-by: Patrick von Platen <[email protected]>

config.json

@@ -5,7 +5,7 @@
     "OPTModel"
   ],
   "attention_dropout": 0.0,
-  "bos_token_id": 0,
+  "bos_token_id": 2,
   "d_model": 768,
   "decoder_layernorm": true,
   "decoder_start_token_id": 2,