up

README.md

@@ -34,8 +34,7 @@ model-index:
 
 # Model Card for Clinical Mosaic
 
-Clinical Mosaic is a transformer-based language model built on the Mosaic BERT architecture. It has been optimized for clinical language understanding by incorporating recent innovations such as Attention with Linear Biases (ALiBi) for long‐sequence extrapolation and Gated Linear Units (GLU) to capture complex clinical patterns. The model is pre-trained on 331,794 deidentified clinical notes from the MIMIC-IV-NOTES 2.2 database and demonstrates strong performance on clinical reasoning tasks.
-
+Clinical Mosaic is a transformer-based language model designed for clinical text, built on the Mosaic BERT architecture. The model is pretrained on 331,794 deidentified clinical notes from the MIMIC-IV-NOTES 2.2 database, with a sequence length of 512 tokens, while leveraging Attention with Linear Biases (ALiBi) to improve extrapolation beyond this limit without re-quiring learned positional embeddings.
 ## Model Details
 - **Developed by:** Sifal Klioui, Sana Sellami, and Youssef Trardi (Aix-Marseille Univ, LIS, CNRS, Marseille, France)
 - **Funded by:** PICOMALE project (AMIDEX) Under the direction of the CEDRE