Fafadalilian/lora-adapter-t5_small_model_California_state_bill

Model Card for Model ID

This model is a fine-tuned version of the T5-small model, enhanced with a LoRA (Low-Rank Adaptation) adapter. It has been specifically fine-tuned to summarize legal documents, focusing on California state bills.

Model Details

Base Model: T5-small Task: Legal Document Summarization (California State Bills) LoRA Configuration: r: 8 lora_alpha: 32 lora_dropout: 0.1 Dataset: "billsum", split="ca_test"

Model Description

• Developed by: Fatemeh Dalilian
• Finetuned from model [optional]: T5-small

Model Sources [optional]

• Repository: [More Information Needed]
• Paper [optional]: [More Information Needed]
• Demo [optional]: [More Information Needed]

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import T5Tokenizer, T5ForConditionalGeneration

# Load the model and tokenizer
tokenizer = T5Tokenizer.from_pretrained("Fafadalilian/lora-adapter-t5_small_model_California_state_bill")
model = T5ForConditionalGeneration.from_pretrained("Fafadalilian/lora-adapter-t5_small_model_California_state_bill")

# Example input text
input_text = "summarize: [Insert California state bill text here]"

# Tokenize the input
inputs = tokenizer(input_text, return_tensors="pt", truncation=True, padding="max_length", max_length=512)

# Generate summary
summary_ids = model.generate(inputs.input_ids, max_length=150, num_beams=2, length_penalty=2.0, early_stopping=True)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)

print("Summary:", summary)