BERT-Base Uncased Fine-Tuned on SQuAD

Overview

This repository contains a BERT-Base Uncased model fine-tuned on the SQuAD (Stanford Question Answering Dataset) for Question Answering (QA) tasks. The model has been fine-tuned for 2 epochs, making it suitable for extracting answers from given contexts by predicting start and end token positions.

The Model predicts 2 probabilities among all the tokens in the vocab , One indicating the start token and the other indicating the end token, Then the answer between both these tokens are extracted.

Model Details

  • Model Type: BERT-Base Uncased
  • Fine-Tuning Dataset: SQuAD (Stanford Question Answering Dataset)
  • Number of Epochs: 2
  • Task: Question Answering
  • Base Model: BERT-Base Uncased

Usage

How to Load the Model

You can load the model using the transformers library from Hugging Face:

from transformers import BertForQuestionAnswering, BertTokenizer

# Load the tokenizer and model
tokenizer = BertTokenizer.from_pretrained("Abdo36/Bert-SquAD-QA")
model = BertForQuestionAnswering.from_pretrained("Abdo36/Bert-SquAD-QA")

context = "BERT is a method of pre-training language representations."
question = "What is BERT?"

inputs = tokenizer.encode_plus(question, context, return_tensors="pt")

# Perform inference
outputs = model(**inputs)
start_scores = outputs.start_logits
end_scores = outputs.end_logits

# Extract answer
start_index = start_scores.argmax()
end_index = end_scores.argmax()
answer = tokenizer.decode(inputs["input_ids"][0][start_index:end_index + 1])

print("Answer:", answer)

Citation

If you use this model in your research, please cite the original BERT paper:

@article{devlin2018bert,
  title={BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding},
  author={Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
  journal={arXiv preprint arXiv:1810.04805},
  year={2018}
}
Downloads last month
21
Safetensors
Model size
109M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train Abdo36/Bert-SquAD-QA