README.md

metadata

library_name: transformers
language:
  - ru
  - en
pipeline_tag: text-generation
license: other
license_name: apache-2.0
license_link: >-
  https://huggingface.co/MTSAIR/Cotype-Nano-GGUF/blob/main/Apache%20License%20MTS%20AI.docx

Cotype Nano GGUF🤖

Cotype-Nano-GGUF– это легковесная LLM, которая может запускаться даже на мобильных девайсах

Cotype-Nano-GGUF is a lightweight LLM that can run even on mobile devices

Inference

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="MTSAIR/Cotype-Nano-GGUF",
    filename="cotype_nano_8bit.gguf",
)

llm.create_chat_completion(
    messages = [
        {
            "role": "user",
            "content": "What is the capital of France?"
        }
    ]
)