Description:

Hello AI4Bharat community,

I’m currently working with the AI4Bharat IndicBERT model and trying to fine-tune it with PEFT (Low-Rank Adaptation) for token classification on a custom dataset. While attempting to load and test the fine-tuned model, I’m encountering issues related to PEFT’s internal configuration loading.

Steps Taken:
Fine-Tuned IndicBERT using my custom dataset for token classification.
Applied LoRA fine-tuning using PEFT to make the model more efficient.
Saved the model and tokenizer correctly.
Tried to load and test the model using the following script (see below).
Problem:
When trying to load and test the fine-tuned model, I get the following error:

Some weights of AlbertForTokenClassification were not initialized from the model checkpoint at ai4bharat/indic-bert and are newly initialized: ['classifier.bias', 'classifier.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
An error occurred: Can't find 'adapter_config.json' at './adapter_config.json'
Traceback (most recent call last):
File "C:\AI_bharat\load_model_test.py", line 13, in
peft_config = PeftConfig.from_pretrained(config_path)
File "C:\AI_bharat\myenv310\lib\site-packages\peft\config.py", line 199, in from_pretrained
raise ValueError(f"Can't find '{CONFIG_NAME}' at '{pretrained_model_name_or_path}'") from exc
ValueError: Can't find 'adapter_config.json' at './adapter_config.json'
The error occurs because PEFT is trying to load adapter_config.json, but I am using a hardcoded LoRA configuration and I have already fine-tuned the model.

Steps I've Tried to Fix the Issue:
I removed any call to PeftConfig.from_pretrained(), as I’m using a manual LoRA configuration.
Manually applied the LoRA configuration using LoraConfig and get_peft_model().
Despite these changes, the error persists because PEFT is still attempting to load the adapter_config.json file.

Questions:
How can I skip PEFT's internal configuration loading to avoid it trying to load adapter_config.json?
Should I disable PEFT's from_pretrained() functionality entirely and manage LoRA manually?
Is there any other method to ensure the fine-tuned model works with LoRA and avoids loading the configuration file?
My Environment:
Python version: 3.10
Libraries:
Transformers: 4.28.0
PEFT: 0.4.0
torch: 2.0.0
ai4bharat/indic-bert: (Specify version or model details)
Error Traceback: (Included above)

Code Snippet:
Here's the script I used to test the model:

python

import json
import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer
from peft import LoraConfig, get_peft_model

Path to my fine-tuned model

model_path = r"C:\AI_bharat\models\finetuned"

Hardcode the adapter configuration (LoRA)

adapter_config = {
"lora_alpha": 32,
"r": 16,
"target_modules": ["value", "query"],
"task_type": "TOKEN_CLS",
"peft_type": "LORA",
"lora_dropout": 0.05,
"lora_bias": False,
}

Initialize LoRA configuration manually

lora_config = LoraConfig(**adapter_config)

Load the fine-tuned model and tokenizer

model = AutoModelForTokenClassification.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)

Apply LoRA fine-tuning manually

model.classifier = torch.nn.Linear(model.config.hidden_size, len(adapter_config['target_modules']))

Manually apply LoRA

model = get_peft_model(model, lora_config)

print("Model and Tokenizer loaded successfully!")

Test with a sample input

test_text = "જાહેર નોટિસ આથી જાહેર જનતાને જણાવવાનું કે, જત ડિસ્ટ્રીકટ સબ ડિસ્ટ્રીકટ અમદાવાદ – ૪ (પાલડી) સાબરમતી તાલુકાના ગામ - મોજે કોચરબની સીમના ટી.પી.સ્કીમ nomb.- ２０ ના ફ.પ્લોટ નં.- ૩૦૮ ની જમીન ઉપર આવેલ સબ પ્લોટ નંબર ૧૨ ની ૪૦૬ સમ ચો.વાર જમીનમાં અવેલા બાંધકામ"
inputs = tokenizer(test_text, return_tensors="pt", padding=True, truncation=True, max_length=512)

Get model outputs

with torch.no_grad():
outputs = model(**inputs)

logits = outputs.logits
predicted_labels = logits.argmax(dim=-1)

List of possible labels

unique_labels_list = ['District', 'Sub-District/Taluka', 'Village/Moje/Gaam', 'NEW Survey/Block No',
'Old Survey/Block No', 'TP No', 'FP No', 'Property Use', 'Property Type',
'Project Name', 'Propery No', 'Advocate Name']

Convert predicted indices to text labels

predicted_label_texts = [unique_labels_list[label.item()] for label in predicted_labels[0]]

print("Predicted Labels:", predicted_label_texts)