davidlvxin commited on
Commit
18b5dbf
•
1 Parent(s): 2210ed7

add readme

Browse files
Files changed (4) hide show
  1. .mdl +0 -0
  2. .msc +0 -0
  3. .mv +0 -1
  4. README.md +50 -0
.mdl DELETED
Binary file (49 Bytes)
 
.msc DELETED
Binary file (1.11 kB)
 
.mv DELETED
@@ -1 +0,0 @@
1
- Revision:master,CreatedAt:1723441815
 
 
README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - zh
5
+ library_name: transformers
6
+ tags:
7
+ - Long Context
8
+ - chatglm
9
+ - llama
10
+ datasets:
11
+ - THUDM/LongWriter-6k
12
+ ---
13
+ # LongWriter-glm4-9b
14
+
15
+ <p align="center">
16
+ 🤗 <a href="https://huggingface.co/datasets/THUDM/LongWriter-6k" target="_blank">[LongWriter Dataset] </a> • 💻 <a href="https://github.com/THUDM/LongWriter" target="_blank">[Github Repo]</a> • 📃 <a href="https://arxiv.org/" target="_blank">[LongWriter Paper]</a>
17
+ </p>
18
+
19
+ LongWriter-glm4-9b is trained based on [glm-4-9b-chat-1m](https://huggingface.co/THUDM/glm-4-9b-chat-1m), and is capable of generating 10,000+ words at once.
20
+
21
+
22
+ A simple demo for deployment of the model:
23
+ ```python
24
+ from transformers import AutoTokenizer, AutoModelForCausalLM
25
+ import torch
26
+ tokenizer = AutoTokenizer.from_pretrained("THUDM/LongWriter-glm4-9b", trust_remote_code=True)
27
+ model = AutoModelForCausalLM.from_pretrained("THUDM/LongWriter-glm4-9b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto")
28
+ model = model.eval()
29
+ query = "Write a 10000-word China travel guide"
30
+ prompt = f"[INST]{query}[/INST]"
31
+ input = tokenizer(prompt, truncation=False, return_tensors="pt").to(device)
32
+ context_length = input.input_ids.shape[-1]
33
+ output = model.generate(
34
+ **input,
35
+ max_new_tokens=32768,
36
+ num_beams=1,
37
+ do_sample=True,
38
+ temperature=0.5,
39
+ )[0]
40
+ response = tokenizer.decode(output[context_length:], skip_special_tokens=True)
41
+ print(response)
42
+ ```
43
+
44
+ ## Citation
45
+
46
+ If you find our work useful, please consider citing LongWriter:
47
+
48
+ ```
49
+
50
+ ```