|
# GPT-Code-Clippy-125M-APPS |
|
|
|
## Model Description |
|
|
|
GPT-CC-125M-APPS is a GPT-Neo-125M finetuned on APPS dataset. This model is specialized to solve programming tasks. |
|
|
|
## Training data |
|
|
|
[APPS dataset](https://github.com/hendrycks/apps). |
|
|
|
## Training procedure |
|
|
|
The training script used to train this model can be found [here](https://github.com/ncoop57/gpt-code-clippy/blob/camera-ready/training/run_clm_apps.py). |
|
|
|
## Intended Use and Limitations |
|
|
|
The model is finetuned to solve programming problems given a text description and optional starter code. |
|
|
|
### How to use |
|
|
|
You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run: |
|
|
|
```py |
|
from transformers import AutoModelForCausalLM, AutoTokenizer, FlaxAutoModelForCausalLM |
|
|
|
model = AutoModelForCausalLM.from_pretrained("flax-community/gpt-code-clippy-125M-apps-alldata") |
|
tokenizer = AutoTokenizer.from_pretrained("flax-community/gpt-code-clippy-125M-apps-alldata") |
|
|
|
prompt = """ |
|
A function to greet user. Given a user name it should say hello |
|
|
|
def greet(name): |
|
|
|
ANSWER: |
|
""" |
|
|
|
input_ids = tokenizer(prompt, return_tensors='pt').input_ids.to(device) |
|
start = input_ids.size(1) |
|
out = model.generate(input_ids, do_sample=True, max_length=50, num_beams=2, |
|
early_stopping=True, eos_token_id=tokenizer.eos_token_id, ) |
|
|
|
print(tokenizer.decode(out[0][start:])) |
|
``` |
|
|
|
### Limitations and Biases |
|
|
|
The model is intended to be used for research purposes and comes with no guarantees of quality of generated code. |
|
GPT-CC is finetuned GPT-Neo and might have inhereted biases and limitations from it. See [GPT-Neo model card](https://huggingface.co/EleutherAI/gpt-neo-125M#limitations-and-biases) for details. |
|
|
|
## Eval results |
|
|
|
Coming soon... |
|
|