MaxJeblick/reward-model-deberta-v3-unit-test

Small dummy deberta-v3-type Reward Model useable for Unit/Integration tests for RLHF. Suitable for CPU only machines, see H2O LLM Studio for an example integration test.

Model was created as follows:

from transformers import AutoConfig, AutoTokenizer, AutoModelForSequenceClassification

repo_name = "MaxJeblick/reward-model-deberta-v3-unit-test"
model_name = "OpenAssistant/reward-model-deberta-v3-large-v2"
config = AutoConfig.from_pretrained(model_name)

config.hidden_size = 12
config.intermediate_size = 24
config.num_attention_heads = 2
config.num_hidden_layers = 2
config.pooler_hidden_size = 12

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForSequenceClassification.from_config(config)
print(model.num_parameters())  # 1_546_129


model.push_to_hub(repo_name, private=False)
tokenizer.push_to_hub(repo_name, private=False)
config.push_to_hub(repo_name, private=False)