This repository contains the TabuLa-8B (Tabular Llama-8B) model. TabuLa-8B is a foundation model for prediction (classification and binned regression) on tabular data.
TabuLa-8B is described in the paper "Large Scale Transfer Learning for Tabular Data via Language Modeling."
For more details on the model, see the paper, which includes a Model Card detailing the model architecture, training, and evaluation. TabuLa-8B was trained with rtfm, using the T4 dataset.
TabuLa-8B is built with Meta Llama 3.
You can load the model with transformers via
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("mlfoundations/tabula-8b")
model = AutoModelForCausalLM.from_pretrained("mlfoundations/tabula-8b")
For more information on how to prepare data and run inference (including a demo notebook for performing inference on your data), see the examples in rtfm.
TabuLa-8B is fine-tuned from the Llama-3 8B model. As a result, we release it under the Llama 3 license, and by using the model you agree to abide by the Llama 3 Community License Agreement and the Llama 3 Acceptable Use Policy.