Cenker Sisman commited on
Commit
85c636c
1 Parent(s): ac847b2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -8
README.md CHANGED
@@ -24,8 +24,7 @@ widget:
24
 
25
  <!-- Provide a quick summary of what the model is/does. -->
26
 
27
- This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).
28
-
29
  ## Model Details
30
 
31
  ### Model Description
@@ -34,12 +33,11 @@ This modelcard aims to be a base template for new models. It has been generated
34
 
35
 
36
 
37
- - **Developed by:** [More Information Needed]
38
- - **Shared by [optional]:** [More Information Needed]
39
  - **Model type:** [More Information Needed]
40
  - **Language(s) (NLP):** [More Information Needed]
41
  - **License:** [More Information Needed]
42
- - **Finetuned from model [optional]:** [More Information Needed]
43
 
44
  ### Model Sources [optional]
45
 
@@ -85,9 +83,41 @@ Users (both direct and downstream) should be made aware of the risks, biases and
85
 
86
  ## How to Get Started with the Model
87
 
88
- Use the code below to get started with the model.
89
-
90
- [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
91
 
92
  ## Training Details
93
 
 
24
 
25
  <!-- Provide a quick summary of what the model is/does. -->
26
 
27
+ Bu model test amaçlı hazırlanmıştır ve fikir vermesi açısından geliştirilmiştir. Model için Vikipedi üzerinden üretilen 40 bin soru cevap GPT ile eğitilmiştir. Daha büyük veri setlerinde daha iyi sonuçlar alınabilir.
 
28
  ## Model Details
29
 
30
  ### Model Description
 
33
 
34
 
35
 
36
+ - **Developed by:** Cenker Sisman
 
37
  - **Model type:** [More Information Needed]
38
  - **Language(s) (NLP):** [More Information Needed]
39
  - **License:** [More Information Needed]
40
+ - **Finetuned from model :** redrussianarmy/gpt2-turkish-cased
41
 
42
  ### Model Sources [optional]
43
 
 
83
 
84
  ## How to Get Started with the Model
85
 
86
+ """Inference"""
87
+
88
+ from transformers import PreTrainedTokenizerFast, GPT2LMHeadModel, GPT2TokenizerFast, GPT2Tokenizer
89
+
90
+ def load_model(model_path):
91
+ model = GPT2LMHeadModel.from_pretrained(model_path)
92
+ return model
93
+
94
+
95
+ def load_tokenizer(tokenizer_path):
96
+ tokenizer = GPT2Tokenizer.from_pretrained(tokenizer_path)
97
+ return tokenizer
98
+
99
+ def generate_text(model_path, sequence, max_length):
100
+ model = load_model(model_path)
101
+ tokenizer = load_tokenizer(model_path)
102
+ ids = tokenizer.encode(sequence, return_tensors='pt')
103
+ outputs = model.generate(
104
+ ids,
105
+ do_sample=True,
106
+ max_length=max_length,
107
+ pad_token_id=model.config.eos_token_id,
108
+ top_k=1,
109
+ top_p=0.99,
110
+ )
111
+ converted = tokenizer.convert_ids_to_tokens(outputs[0])
112
+ valid_tokens = [token if token is not None else '.' for token in converted]
113
+ generated_text = tokenizer.convert_tokens_to_string(valid_tokens)
114
+ print(generated_text)
115
+
116
+ model2_path = "Cenker/turkishgpt"
117
+ sequence2 = "[Q] cengiz han kimdir"
118
+ max_len = 120
119
+
120
+ generate_text(model2_path, sequence2, max_len)
121
 
122
  ## Training Details
123