QuantFactory Banner

QuantFactory/Typst-Coder-9B-GGUF

This is quantized version of TechxGenus/Typst-Coder-9B created using llama.cpp

Original Model Card

Typst-Coder

[🤖Models] | [🛠️Code] | [📊Data] |



Introduction

While working with Typst documents, we noticed that AI programming assistants often generate poor results. I understand that these assistants may perform better in languages like Python and JavaScript, which benefit from more extensive datasets and feedback signals from executable code, unlike HTML or Markdown. However, current LLMs even frequently struggle to produce accurate Typst syntax, including models like GPT-4o and Claude-3.5-Sonnet.

Upon further investigation, we found that because Typst is a relatively new language, training data for it is scarce. GitHub's search tool doesn't categorize it as a language for code yet, and The Stack v1/v2 don’t include Typst. No open code LLMs currently list it as a supported language, either. To address this, we developed this project aimed at collecting relevant data and training models to improve Typst support in AI programming tools.

Usage

An example script is shown below:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TechxGenus/Typst-Coder-9B")
model = AutoModelForCausalLM.from_pretrained(
    "TechxGenus/Typst-Coder-9B",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "user", "content": "Hi!"},
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer.encode(prompt, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=512)
print(tokenizer.decode(outputs[0]))
Downloads last month
236
GGUF
Model size
8.83B params
Architecture
llama

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for QuantFactory/Typst-Coder-9B-GGUF

Base model

01-ai/Yi-Coder-9B
Quantized
(11)
this model