Model Card for Walid-Ahmed/finetuned-falcon-medtext
This model is a fine-tuned version of the Falcon-7B model, trained on a medical question-answering dataset to generate medically relevant text. It uses causal language modeling and was fine-tuned to better understand structured queries and answers in the healthcare domain.
Model Details
Model Description
This model adapts the Falcon-7B architecture for domain-specific use in medical dialogue and question answering. It was trained using the Hugging Face Trainer with causal language modeling on a dataset that combines patient prompts and medically appropriate responses.
- Developed by: Walid Ahmed
- Model type: Causal Language Model
- Language(s): English
- License: Apache 2.0
- Finetuned from: tiiuae/falcon-7b
Model Sources
Uses
Direct Use
This model can be used as a text-generation tool for medical Q&A or dialogue simulation, particularly for research, academic, and educational purposes.
Example Prompt:
A 25-year-old female presents with swelling, pain, and inability to bear weight on her left ankle following a fall during a basketball game where she landed awkwardly on her foot. The pain is on the outer side of her ankle. What is the likely diagnosis and next steps?
Expected Output
The model will generate a medically plausible diagnosis (e.g., lateral ankle sprain or fracture) and suggest typical next steps such as physical examination, Ottawa ankle rules assessment, or imaging (e.g., ankle X-ray) based on clinical best practices.
Downstream Use
- Educational tools in clinical reasoning
- Prompt-based evaluation research
- Foundation for further fine-tuning in healthcare
Out-of-Scope Use
- Real-world diagnosis or treatment
- Use in clinical settings
- Any patient-facing applications without human supervision
Bias, Risks, and Limitations
- May hallucinate or generate incorrect medical information
- Model is not verified by medical professionals
- Trained data may reflect bias or outdated practices
Recommendations
- Strictly for educational and research use
- Avoid real-world clinical reliance without oversight
How to Get Started with the Model
from transformers import pipeline
pipe = pipeline("text-generation", model="Walid-Ahmed/finetuned-falcon-medtext")
prompt = (
"A 25-year-old female presents with swelling, pain, and inability to bear weight "
"on her left ankle following a fall during a basketball game where she landed awkwardly "
"on her foot. The pain is on the outer side of her ankle. What is the likely diagnosis "
"and next steps?"
)
output = pipe(prompt, max_new_tokens=100, do_sample=True, temperature=0.7)
print(output[0]["generated_text"])
Example Output:
The likely diagnosis is a lateral ankle sprain or a possible fracture of the fibula, such as a distal fibular fracture. The next steps include a physical examination using the Ottawa ankle rules and ordering an X-ray of the ankle to rule out a fracture. Initial treatment should involve R.I.C.E. (Rest, Ice, Compression, Elevation) and pain control.
Training Details
Training Data
This model was fine-tuned on the BI55/MedText dataset.
The dataset contains over 1,000 high-quality clinical scenarios, representing textbook-level patient presentations across:
- The 100 most common diseases
- The 30 most common hospital-presenting injuries
- Scenarios ranging from mild to severe
The dataset also includes:
- Scenarios where the AI should not answer due to ambiguity or lack of data
- Ethical messaging reminding users that AI cannot replace a human doctor
- Completions that encourage seeking professional medical advice
Sample Prompt & Completion
Prompt:
A 50-year-old male presents with a history of recurrent kidney stones and osteopenia. He has been taking high-dose vitamin D supplements due to a previous diagnosis of vitamin D deficiency. Laboratory results reveal hypercalcemia and hypercalciuria. What is the likely diagnosis, and what is the treatment?
Completion:
This patient's history of recurrent kidney stones, osteopenia, and high-dose vitamin D supplementation, along with laboratory findings of hypercalcemia and hypercalciuria, suggest the possibility of vitamin D toxicity. Excessive intake of vitamin D can cause increased absorption of calcium from the gut, leading to hypercalcemia and hypercalciuria, which can result in kidney stones and bone loss. Treatment would involve stopping the vitamin D supplementation and potentially providing intravenous fluids and loop diuretics to promote the excretion of calcium.
Training Procedure
Preprocessing
- Tokenized using Falcon-7B tokenizer
- Formatted as single-sequence CLM tasks
- Max input length: 1024 tokens
Training Hyperparameters
from transformers import TrainingArguments
training_args = TrainingArguments(
output_dir="./finetuned_falcon",
eval_strategy="epoch",
learning_rate=2e-5,
weight_decay=0.01,
fp16=True,
per_device_train_batch_size=1,
per_device_eval_batch_size=1,
gradient_accumulation_steps=1,
logging_steps=1,
num_train_epochs=1,
optim="paged_adamw_8bit",
report_to="none"
)
Speeds, Sizes, Times
- Training duration: ~[fill in]
- Model size: ~13.4 GB (fp16)
- Hardware used: e.g., 1× A100 80GB
Evaluation
Testing
Manual inspection of unseen medical prompts.
Focus: Fluency, correctness, and relevance.
Metrics
No automated evaluation metrics reported.
Qualitative review confirms accurate basic clinical reasoning.
Environmental Impact
- Hardware Type: NVIDIA A100
- Hours used: [estimated]
- Compute Region: [Optional]
- Carbon Emitted: Estimate using ML CO2 calculator
Technical Specifications
- Architecture: Falcon-7B
- Objective: Causal Language Modeling (CLM)
- Precision: fp16
- Training Framework: Hugging Face Transformers
- Optimizer: 8-bit AdamW (paged)
Citation
BibTeX:
@misc{falcon-medtext,
author = {Walid Ahmed},
title = {Fine-Tuned Falcon-7B for Medical QA},
year = {2025},
url = {https://huggingface.co/Walid-Ahmed/finetuned-falcon-medtext}
}
APA:
Ahmed, W. (2025). Fine-Tuned Falcon-7B for Medical QA. Hugging Face. https://huggingface.co/Walid-Ahmed/finetuned-falcon-medtext
Contact
Author: Walid Ahmed
Email: [email protected]
Date: April 17, 2025
- Downloads last month
- 54