bofenghuang commited on
Commit
d9171c5
1 Parent(s): 4bd59ac

Update readme

Browse files
Files changed (1) hide show
  1. README.md +36 -3
README.md CHANGED
@@ -22,15 +22,48 @@ For more information, please visit the Github repo: https://github.com/bofenghua
22
 
23
  **Usage and License Notices**: Same as [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca), Vigogne is intended and licensed for research use only. The dataset is CC BY NC 4.0 (allowing only non-commercial use) and models trained using the dataset should not be used outside of research purposes.
24
 
25
- ## Usage
 
 
 
 
 
 
 
26
 
27
- This repo only contains the low-rank adapter. In order to access the complete model, you also need to load the base LLM model and tokenizer.
28
 
29
  ```python
 
 
 
 
 
 
 
 
 
 
 
 
30
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  ```
32
 
33
- You can infer this model by using the following Google Colab Notebook.
34
 
35
  <a href="https://colab.research.google.com/github/bofenghuang/vigogne/blob/main/notebooks/infer_instruct.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
36
 
 
22
 
23
  **Usage and License Notices**: Same as [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca), Vigogne is intended and licensed for research use only. The dataset is CC BY NC 4.0 (allowing only non-commercial use) and models trained using the dataset should not be used outside of research purposes.
24
 
25
+ ## Changelog
26
+
27
+ All versions are available in branches.
28
+
29
+ - v1.0: Initial release, trained on the translated Stanford Alpaca dataset.
30
+ - v1.1: Improved translation quality of the Stanford Alpaca dataset.
31
+ - v2.0: Expanded training dataset to 224k for better performance.
32
+ - v3.0: Further expanded training dataset to 262k for improved results.
33
 
34
+ ## Usage
35
 
36
  ```python
37
+ import torch
38
+ from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
39
+ from vigogne.preprocess import generate_instruct_prompt
40
+
41
+ model_name_or_path = "bofenghuang/vigogne-7b-instruct"
42
+ tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, padding_side="right", use_fast=False)
43
+ model = AutoModelForCausalLM.from_pretrained(model_name_or_path, torch_dtype=torch.float16, device_map="auto")
44
+
45
+ user_query = "Expliquez la différence entre DoS et phishing."
46
+ prompt = generate_instruct_prompt(user_query)
47
+ input_ids = tokenizer(prompt, return_tensors="pt")["input_ids"].to(model.device)
48
+ input_length = input_ids.shape[1]
49
 
50
+ generated_outputs = model.generate(
51
+ input_ids=input_ids,
52
+ generation_config=GenerationConfig(
53
+ temperature=0.1,
54
+ do_sample=True,
55
+ repetition_penalty=1.0,
56
+ #no_repeat_ngram_size=no_repeat_ngram_size,
57
+ max_new_tokens=512,
58
+ ),
59
+ return_dict_in_generate=True,
60
+ )
61
+ generated_tokens = generated_outputs.sequences[0, input_length:]
62
+ generated_text = tokenizer.decode(generated_tokens, skip_special_tokens=True)
63
+ print(generated_text)
64
  ```
65
 
66
+ You can also infer this model by using the following Google Colab Notebook.
67
 
68
  <a href="https://colab.research.google.com/github/bofenghuang/vigogne/blob/main/notebooks/infer_instruct.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
69