afrizalha
/

Kancil-V1-llama3-4bit

@@ -1,23 +1,124 @@
 ---
-language:
-- en
-license: apache-2.0
 tags:
-- text-generation-inference
-- transformers
 - unsloth
-- llama
-- trl
-- sft
-base_model: unsloth/llama-3-8b-bnb-4bit
 ---
-# Uploaded  model
-- **Developed by:** catinthebag
-- **License:** apache-2.0
-- **Finetuned from model :** unsloth/llama-3-8b-bnb-4bit
-This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
-[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

 ---
+library_name: transformers
 tags:
 - unsloth
+- llama3
+- indonesia
+license: llama3
+datasets:
+- catinthebag/TumpengQA
+language:
+- id
+inference: false
 ---
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Document Title</title>
+    <style>
+        h1 {
+            font-size: 36px;
+            color: navy;
+            font-family: 'Tahoma';
+            text-align: center;
+        }
+    </style>
+</head>
+<body>
+    <h1>Introducing the Kancil family of open models</h1>
+</body>
+</html>
+<center>
+    <img src="https://imgur.com/9nG5J1T.png" alt="Kancil" width="600" height="300">
+    <p><em>Kancil is a fine-tuned version of Llama 3 8B using synthetic QA dataset generated with Llama 3 70B. Version zero of Kancil is the first generative Indonesian LLM gain functional instruction performance using solely synthetic data.</em></p>
+    <p><strong><a href="https://colab.research.google.com/drive/1526QJYfk32X1CqYKX7IA_FFcIHLXbOkx?usp=sharing" style="color: blue; font-family: Tahoma;">❕Go straight to the colab demo❕</a></strong></p>
+    <p><em style="color: seagreen;">Compatibility erorrs are fixed! The colab should work fine now.</em></p>
+</center>
+Selamat datang!
+I am ultra-overjoyed to introduce you... the 🦌 Kancil! It's a fine-tuned version of Llama 3 8B with the Tumpeng, an instruction dataset of 6.7 million words. Both the model and dataset is openly available in Huggingface.
+📚 The dataset was synthetically generated from Llama 3 70B. A big problem with existing Indonesian instruction dataset is they're in reality not-very-good-translations of English datasets. Llama 3 70B can generate fluent Indonesian! (with minor caveats 😔)
+🦚 This follows previous efforts for collection of open, fine-tuned Indonesian models, like Merak and Cendol. However, Kancil solely leverages synthetic data in a very creative way, which makes it a very unique contribution!
+### Version 1.0
+This is the second working prototype, Kancil V1.
+✨ Training
+- 2.2x Dataset word count
+- 2x lora parameters
+- Rank-stabilized lora
+- 2x fun
+✨ New features
+- Multi-turn conversation (beta; optimized for curhat/personal advice 😂)
+- Better text generation (full or outline writing; optimized for essays)
+- QA from text (copy paste to prompt and ask a question about it)
+- Making slogans
+This model was fine-tuned with QLoRA using the amazing Unsloth framework! It was built on top of [unsloth/llama-3-8b-bnb-4bit](https://huggingface.co/unsloth/llama-3-8b-bnb-4bit) and subsequently merged with the adapter back to 4 bit (no visible difference with merging back to fp 16).
+### Uses
+This model is developed with research purposes for  researchers or general AI hobbyists. However, it has one big application: You can have lots of fun with it!
+### Out-of-Scope Use
+This is a research preview model with minimal safety curation. Do not use this model for commercial or practical applications.
+You are also not allowed to use this model without having fun.
+### Getting started
+As mentioned, this model was trained with Unsloth. Please use its code for better experience.
+```
+# Install dependencies. You need GPU to run this (at least T4)
+%%capture
+!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
+!pip install --no-deps xformers trl peft accelerate bitsandbytes
+from unsloth import FastLanguageModel
+import torch
+# Available versions
+KancilV1 = "catinthebag/Kancil-V1-llama3-4bit"
+# Load the model
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name = KancilV1,
+    max_seq_length = 4096,
+    dtype = None, # Auto detect
+    load_in_4bit = True,
+)
+```
+```
+# This model was trained on this specific prompt template. Changing it might lead to performance degradations.
+prompt_template = """<|user|>
+{prompt}
+<|assistant|>
+{response}"""
+# Start generating!
+inputs = tokenizer(
+[
+prompt_template.format(
+        prompt="""Bagaimana cara memberi tahu orang tua kalau saya ditolak universitas favorit saya?""",
+        response="",)
+], return_tensors = "pt").to("cuda")
+outputs = model.generate(**inputs, max_new_tokens = 600, temperature=.3, use_cache = True)
+print(tokenizer.batch_decode(outputs)[0].replace('\\n', '\n'))
+```
+**Note:** There is an issue with the dataset where the newline characters are interpreted as literal strings. Very sorry about this! 😔 Please keep the .replace() method to fix newline errors.
+### Acknowledgments
+- **Developed by:** Afrizal Hasbi Azizy
+- **License:** Llama 3 Community License Agreement