Model Card for GPT_2_CODE
WIP, Goal is to create a small GPT2 python coder
Table of Contents
- Model Card for GPT_2_CODE
- Table of Contents
- Table of Contents
- Model Details
- Uses
- Bias, Risks, and Limitations
- Training Details
- Evaluation
- Model Examination
- Environmental Impact
- Technical Specifications [optional]
- Citation
- Glossary [optional]
- More Information [optional]
- Model Card Authors [optional]
- Model Card Contact
- How to Get Started with the Model
Model Details
Model Description
WIP, Goal is to create a small GPT2 python coder
- Developed by: C, o, d, e, M, o, n, k, e, y
- Shared by [Optional]: More information needed
- Model type: Language model
- Language(s) (NLP): eng
- License: wtfpl
- Parent Model: More information needed
- Resources for more information: More information needed
Uses
Direct Use
generate python code snippets
Downstream Use [Optional]
semi auto coder
Out-of-Scope Use
describe code
Bias, Risks, and Limitations
Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)). Predictions generated by the model may include disturbing and harmful stereotypes across protected classes; identity characteristics; and sensitive, social, and occupational groups.
Recommendations
Keep Finetuning on question/python datasets
Training Details
Training Data
flytech/python-codes-25k espejelomar/code_search_net_python_10000_examples
Training Procedure
Preprocessing
More information needed
Speeds, Sizes, Times
Epochs 3 flytech/python-codes-25k (4600) Training Loss: 0.4007 Validation Loss: 0.5526
Epochs 3 espejelomar/code_search_net_python_10000_examples (4800) Training Loss: 1.5355 Validation Loss: 1.1723
Evaluation
Testing Data, Factors & Metrics
Testing Data
flytech/python-codes-25k espejelomar/code_search_net_python_10000_examples
Factors
80/20 train/val
Metrics
train/validate lr scheduling
Results
Better in python code generation as base gpt2-medium model
Model Examination
More information needed
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: CPU and Colab T4
- Hours used: 4
- Cloud Provider: Google Colab
- Compute Region: NL
- Carbon Emitted: ???
Technical Specifications [optional]
Model Architecture and Objective
gpt2
Compute Infrastructure
More information needed
Hardware
CPU and Colab T4
Software
pytorch, custom python
Citation
BibTeX:
More information needed
APA:
More information needed
Glossary [optional]
More information needed
More Information [optional]
Experimental
Model Card Authors [optional]
CodeMonkeyXL
Model Card Contact
K00B404 huggingface
How to Get Started with the Model
Use the code below to get started with the model.
Click to expand
More information needed