File size: 1,208 Bytes
2702ca2 928fed6 2702ca2 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
---
language: protein
tags:
- protein
datasets:
- uniref-100
---
# RITA-XL
RITA is a family of autoregressive protein models, developed in collaboration between Lighton, Harvard and Oxford.
Model | #Params | d_model | layers | lm loss uniref-100
--- | --- | --- | --- | --- |
[Small](https://huggingface.co/lightonai/RITA_s) | 85M | 768 | 12 | 2.31
[Medium](https://huggingface.co/lightonai/RITA_m) | 300M | 1024 | 24 | 2.01
[Large](https://huggingface.co/lightonai/RITA_l)| 680M | 1536 | 24 | 1.82
[**XLarge**](https://huggingface.co/lightonai/RITA_xl)| 1.2B | 2048 | 24 | 1.70
# Usage
Instantiate a model like so:
from transformers import AutoModel, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Seledorn/RITA_xl, trust_remote_code=True")
tokenizer = AutoTokenizer.from_pretrained("Seledorn/RITA_xl")
for generation use we support pipelines:
rita_gen = pipeline('text-generation', model=model, tokenizer = tokenizer)
sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, num_return_sequences=2, eos_token_id=2)
for seq in sequences:
print(f"seq: {seq['generated_text'].replace(' ', '')}")
|