mihaimasala commited on
Commit
01e74b2
·
verified ·
1 Parent(s): 8194571

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +87 -0
README.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ language:
4
+ - ro
5
+ ---
6
+
7
+ # Model Card for Model ID
8
+
9
+ <!-- Provide a quick summary of what the model is/does. -->
10
+
11
+ RoLlama2 is a family of pretrained and fine-tuned generative text models for Romanian. This is the repository for the foundational 7B models. Links to other models can be found at the bottom of this page.
12
+
13
+ ## Model Details
14
+
15
+ ### Model Description
16
+
17
+ <!-- Provide a longer summary of what this model is. -->
18
+ RoLlama 2 represents the first open-source effort to build a LLM specialized for Romanian. OpenLLM-Ro developed and publicly releases a collection of Romanian LLMs, both in the form of foundational model and instruct and chat variants.
19
+
20
+
21
+ - **Developed by:** OpenLLM-Ro
22
+ <!-- - **Funded by [optional]:** [More Information Needed] -->
23
+ <!-- - **Shared by [optional]:** [More Information Needed] -->
24
+ <!-- - **Model type:** [More Information Needed] -->
25
+ - **Language(s):** Romanian
26
+ - **License:** cc-by-nc-4.0
27
+ <!-- - **Finetuned from model [optional]:** [More Information Needed] -->
28
+
29
+ ### Model Sources
30
+
31
+ <!-- Provide the basic links for the model. -->
32
+
33
+ - **Repository:** https://github.com/OpenLLM-Ro/llama-recipes
34
+ - **Paper:** [More Information Needed]
35
+
36
+ ## Intended Use
37
+
38
+ ### Intended Use Cases
39
+
40
+ RoLlama2is intented for research use in Romanian. Base models can be adapted for a variety of natural language tasks while instruction and chat tuned models are intended for assistant-like chat.
41
+
42
+ ### Out-of-Scope Use
43
+
44
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
45
+
46
+ Use in any manner that violates the license, any applicable laws or regluations, use in languages other than Romanian.
47
+
48
+
49
+
50
+ ## How to Get Started with the Model
51
+
52
+ Use the code below to get started with the model.
53
+
54
+ ```python
55
+ from transformers import AutoTokenizer, AutoModelForCausalLM
56
+
57
+ tokenizer = AutoTokenizer.from_pretrained("OpenLLM-Ro/RoLlama2-7b-Base")
58
+ model = AutoModelForCausalLM.from_pretrained("OpenLLM-Ro/RoLlama2-7b-Base")
59
+
60
+ input_text = "Write me a poem about Machine Learning."
61
+ input_ids = tokenizer(input_text, return_tensors="pt")
62
+
63
+ outputs = model.generate(**input_ids)
64
+ print(tokenizer.decode(outputs[0]))
65
+ ```
66
+
67
+ ## Benchmarks
68
+
69
+ | Model | Average | ARC | MMLU |Winogrande|HellaSwag | GSM8k |TruthfulQA|
70
+ |--------------------|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
71
+ | Llama-2-7b | 35.65 | 33.85 | 30.93 | 56.43 | 46.98 | 1.37 | 44.36 |
72
+ | *RoLlama2-7b-Base* | *38.32* | *35.83* | *30.47* | *60.16* | *55.52* | *2.17* | *45.78* |
73
+ | Llama-2-7b-chat | 35.58 | 34.92 | 32.37 | 54.26 | 44.52 | 2.05 | 45.38 |
74
+ |RoLlama2-7b-Instruct| **44.42**|**40.36** |**37.41** |**69.58** | 55.64 | **17.59**| 45.96 |
75
+ |RoLlama2-7b-Chat | 42.65 | 38.29 | 35.27 | 65.25 | **56.45**| 12.84 | **47.79**|
76
+
77
+ ## Citation
78
+
79
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
80
+
81
+ **BibTeX:**
82
+
83
+ [More Information Needed]
84
+
85
+ **APA:**
86
+
87
+ [More Information Needed]