Model Card for LLaMat-3

LLaMat-3 is a specialized large language model designed to be a foundational large language model for materials science.

---

Overview

• Model Type: Large Language Model (LLM)
• Base Model: LLaMat-3 (continued pretraining of LLaMA-3 on material science data)
• Language: English
• License: LLaMA-3 License
• Tags: Material Science, Domain Adaptation, Table Understanding, Scientific Data Parsing, Materials Copilot

---

Model Details

Key Features

• Applications: Can be finetuned for information extraction, table understanding, parsing data for research tasks, and crystal structure generation.

Development and Support

• Developed by: M3RG, IIT Delhi & DAIR, IIT Delhi
• Compute Support:
  • Edinburgh International Data Facility (EIDF): Provided access to Cerebras CS2 clusters for pretraining.
  • IIT Delhi High-Performance Computing Cluster: Supported fine-tuning and inference stages.

---

Technical Specifications

Hardware Infrastructure

• Pretraining: 2 Cerebras CS-2 Wafer-Scale Engines (WSE-2)

Software Stack

• Frameworks: PyTorch, Hugging Face Transformers

---

Model Sources

• Repository: LLaMat on GitHub
• Compute Resources: EIDF Cerebras CS Clusters

---