Edit model card

Model Card for Model ID

This model builds upon PepMLM, aimed at generating peptides from receptor sequences. It incorporates the ESM model framework from HuggingFace for its core architecture. The key enhancement in this model is the adoption of LoRA for training, distinguishing it from its predecessor.

Usage:

from transformers import AutoTokenizer, AutoModelForMaskedLM
from peft import PeftConfig
import torch
model_name = "littleworth/esm2_t33_650M_UR50D_pepmlm_lora_adapter_merged"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForMaskedLM.from_pretrained(model_name).to(device)

Packages version:

{'transformers': '4.36.0', 'peft': '0.9.0', 'torch': '2.0.0'}

Training summary:

{
  "train/loss": 1.5091,
  "train/grad_norm": 3.6427412033081055,
  "train/learning_rate": 6.773224309612687e-7,
  "train/epoch": 5,
  "train/global_step": 6395,
  "_timestamp": 1709229361.5373268,
  "_runtime": 25556.57973074913,
  "_step": 639,
  "train/train_runtime": 25557.6176,
  "train/train_samples_per_second": 4.003,
  "train/train_steps_per_second": 0.25,
  "train/total_flos": 220903283526564960,
  "train/train_loss": 1.8436848362317955,
  "_wandb": {
    "runtime": 25556
  }
}
Downloads last month
0
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for littleworth/esm2_t33_650M_UR50D_pepmlm_lora_adapter_merged

Adapter
(1)
this model