## GPT Neo 125m fine-tuned

#### Pushing model to repo

1. Login to hugging face,

```
from huggingface_hub import notebook_login
notebook_login()
```
2. Then push model to repo.
```
model.push_to_hub("gpt-neo-125m-finetuned", use_temp_dir=True)
tokenizer.push_to_hub("gpt-neo-125m-finetuned", use_temp_dir=True)
```

---
#### Using the Model
Load the model along with the tokenizer:
```
tokenizer = GPT2Tokenizer.from_pretrained("ytling/gpt-neo-125m-finetuned", bos_token='<|startoftext|>',eos_token='<|endoftext|>', pad_token='<|pad|>')
gpt_model = GPTNeoForCausalLM.from_pretrained("ytling/gpt-neo-125m-finetuned").cuda()
gpt_model.resize_token_embeddings(len(tokenizer))
```

To use model, pass text, the loaded model and tokenizer into the `gpt_model()` function, 
```
def gpt_model(block_text, model, tokenizer):
  block_dict = { 
        # add labels here
        "Use Case":None
        }
  for label in block_dict:
      prompt = f"<|startoftext|>Text: {block_text}\n{label}: "
      token_prompt = tokenizer(f"{prompt}", return_tensors='pt', padding=True).input_ids.cuda()

      output = model.generate(token_prompt, do_sample=False, top_k=50, max_length=512, top_p=0.80, 
                              temperature=1.08, num_return_sequences=1, pad_token_id=tokenizer.pad_token_id)

      decode_output = tokenizer.decode(output[0], skip_special_tokens=True)
      try:
          block_dict[label] = re.findall(f"\n{label}: (.*)", decode_output)[-1]
      except:
          pass
          
  return block_dict
```

returns dict containing predicted entities within text. 
```
# Eg. 
{'Use Case': "'unify contact centre, unified communications, and real-time communications API capabilities within a single software solution.'"}
```