ritabratamaiti commited on
Commit
8301727
1 Parent(s): 2258e59

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +179 -0
README.md ADDED
@@ -0,0 +1,179 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-sa-4.0
3
+ datasets:
4
+ - Xilabs/PIPPA-alpaca
5
+ language:
6
+ - en
7
+ pipeline_tag: text-generation
8
+ ---
9
+
10
+ # Calypso 3B - Alpha V2 Model Card
11
+
12
+ ## Model Description
13
+
14
+ **Model Name:** Calypso 3B
15
+ **Version:** Calypso 3B - Alpha V2
16
+ **Based on:** [openlm-research/open_llama_3b_v2](https://huggingface.co/openlm-research/open_llama_3b_v2)
17
+
18
+ Calypso 3B is a language model designed for one-on-one chat interactions with a character or persona. It has been finetuned on the PIPPA-Alpaca dataset and a private dataset of human-generated chats. The model is particularly suuited for providing conversational responses in a variety of contexts, making it suitable for role-playing, or one-on-one chatting.
19
+
20
+ ## Intended Use
21
+
22
+ Calypso 3B is intended to facilitate engaging and interactive one-on-one chat experiences.
23
+
24
+ ## Limitations and Ethical Considerations
25
+
26
+ - **Safety Note:** Calypso 3B can produce content that may not be safe for all audiences. It may generate inappropriate, offensive, or sensitive content. User discretion is advised.
27
+
28
+ - **Factual Accuracy:** The model's responses may not always be factually accurate. It should not be relied upon to provide accurate information, especially in critical or sensitive contexts.
29
+
30
+ - **Bias and Fairness:** As with many language models, Calypso 3B might inadvertently exhibit biases present in the training data. Efforts have been made to mitigate this, but biases may still be present.
31
+
32
+ ## Example Usage
33
+
34
+ ```python
35
+ import gradio as gr
36
+ from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig
37
+ import torch
38
+ from transformers import LlamaForCausalLM, LlamaTokenizer
39
+
40
+
41
+ class Chat:
42
+ def __init__(self, model, tokenizer, conv_prompt, user_alias='User', character_name='Chatbot', message_history=[], chat_buffer_size=10):
43
+ self.model = model
44
+ self.tokenizer = tokenizer
45
+ self.conv_prompt = conv_prompt
46
+ self.user_alias = user_alias
47
+ self.character_name = character_name
48
+ self.chat_buffer_size = chat_buffer_size
49
+ self.message_history = message_history
50
+ self.display_messages = []
51
+ for message_pairs in message_history:
52
+ message1, message2 = message_pairs
53
+ self.display_messages.append([message1['text'], message2['text']])
54
+
55
+ def evaluate(self, message, temperature=0.6, top_p=0.75, top_k=50, num_beams=5, max_new_tokens=256, repetition_penalty=1.4, **kwargs):
56
+ prompt = self.prompt_gen_chat(self.message_history, message)
57
+ inputs = self.tokenizer(prompt, return_tensors="pt")
58
+ input_ids = inputs["input_ids"].to(self.model.device)
59
+ generation_config = GenerationConfig(
60
+ temperature=temperature,
61
+ top_p=top_p,
62
+ top_k=top_k,
63
+ num_beams=num_beams,
64
+ early_stopping=True,
65
+ repetition_penalty=repetition_penalty,
66
+ **kwargs,
67
+ )
68
+ with torch.no_grad():
69
+ generation_output = self.model.generate(
70
+ input_ids=input_ids,
71
+ generation_config=generation_config,
72
+ return_dict_in_generate=True,
73
+ output_scores=True,
74
+ max_new_tokens=max_new_tokens,
75
+ )
76
+ s = generation_output.sequences[0]
77
+ output = self.tokenizer.decode(s, skip_special_tokens=True)
78
+ split_str = """### Response:\n{self.character_name}:"""
79
+ output = output.split(split_str)[1].strip()
80
+ return output
81
+
82
+ def gradio_helper(self, message):
83
+ # make response
84
+ response = self.evaluate(message)
85
+ # update message history
86
+ self.message_history.append(
87
+ (
88
+ {"speaker": self.user_alias, "text": message},
89
+ {"speaker": self.character_name, "text": response},
90
+ )
91
+ )
92
+ if len(self.message_history) > self.chat_buffer_size:
93
+ self.message_history = self.message_history[-self.chat_buffer_size:]
94
+ # update display messages
95
+ self.display_messages.append([message, response])
96
+ return self.display_messages
97
+
98
+ def prompt_gen_chat(self, message_history, message):
99
+ past_dialogue = []
100
+ for message_pairs in message_history:
101
+ message1, message2 = message_pairs
102
+ past_dialogue.append(f"{message1['speaker']}: {message1['text']}")
103
+ past_dialogue.append(f"{message2['speaker']}: {message2['text']}")
104
+ past_dialogue_formatted = "\n".join(past_dialogue)
105
+
106
+ prompt = f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
107
+
108
+ ### Instruction:
109
+ {self.conv_prompt}
110
+
111
+ This is the conversation between {self.user_alias} and {self.character_name} till now:
112
+ {past_dialogue_formatted}
113
+
114
+ Continuing from the previous conversation, write what {self.character_name} says to {self.user_alias}:
115
+ ### Input:
116
+ {self.user_alias}: {message}
117
+ ### Response:
118
+ {self.character_name}:"""
119
+
120
+ return prompt
121
+
122
+ def launch_gradio(self):
123
+ with gr.Blocks(theme="JohnSmith9982/small_and_pretty") as demo:
124
+ chatbot = gr.Chatbot(elem_id="chatbot")
125
+ with gr.Row():
126
+ txt = gr.Textbox(show_label=False,
127
+ placeholder="Enter text and press enter")
128
+ txt.submit(self.gradio_helper, txt, chatbot)
129
+ txt.submit(lambda: "", None, txt)
130
+
131
+ demo.launch(debug=True, share=True)
132
+
133
+
134
+ if __name__ == "__main__":
135
+ model_path = "Xilabs/calypso-3b-alpha-v2"
136
+ load_in_8bit = False
137
+ model = LlamaForCausalLM.from_pretrained(
138
+ model_path, device_map="auto", load_in_8bit=load_in_8bit)
139
+ tokenizer = LlamaTokenizer.from_pretrained(model_path)
140
+ conv_prompt = "Two people are texting each other on a messaging platform."
141
+ message_history = [
142
+ (
143
+ {
144
+ "speaker": "Bob",
145
+ "text": "Hey, Alice! How are you doing? What's the status on those reports?",
146
+ },
147
+ {
148
+ "speaker": "Alice",
149
+ "text": "Hey, Bob! I'm doing well. I'm almost done with the reports. I'll send them to you by the end of the day.",
150
+ },
151
+ ),
152
+ (
153
+ {
154
+ "speaker": "Bob",
155
+ "text": "That's great! Thanks, Alice. I'll be waiting for them. Btw, I have approved your leave for next week.",
156
+ },
157
+ {
158
+ "speaker": "Alice",
159
+ "text": "Oh, thanks, Bob! I really appreciate it. I will be sure to send you the reports before I leave. Anything else you need from me?",
160
+ },
161
+ )
162
+ ]
163
+
164
+ chat_instance = Chat(model, tokenizer, conv_prompt, user_alias='Bob',
165
+ character_name='Alice', message_history=message_history)
166
+ chat_instance.launch_gradio()
167
+ ```
168
+
169
+ ## Future Improvements
170
+
171
+ Calypso 3B is an ongoing project, and future iterations will focus on enhancing safety, improving factual accuracy, and reducing biases in its responses. The development team is committed to addressing user feedback and continuously improving the model's performance.
172
+
173
+ ## Licensing and Commercial Use
174
+
175
+ Larger and more permissive versions of Calypso will be released in the future. If you're interested in using Calypso 3B or its future iterations for commercial purposes, obtaining a license, or accessing the model via an API, please reach out to us for more information.
176
+
177
+ ---
178
+
179
+ **Disclaimer:** This model card is provided for informational purposes only. Users are responsible for using the model in accordance with applicable laws and ethical considerations.