|
--- |
|
language: fr |
|
license: mit |
|
tags: |
|
- causal-lm |
|
- fr |
|
datasets: |
|
- gustavecortal/fr_covid_news |
|
--- |
|
|
|
### GPT-J COVID-19 French News with 8-bit weights |
|
|
|
|
|
This is a version of Cedille's GPT-J ([fr-boris](https://huggingface.co/gustavecortal/fr-boris-8bit)) with 6 billion parameters fine-tuned on [COVID-19 French News dataset](https://huggingface.co/datasets/gustavecortal/fr_covid_news) to generate French headlines related to COVID-19. |
|
|
|
You can generate the model in colab or equivalent desktop gpu (e.g. single 1080Ti) as the model has 8-bit weights. Inspired by [GPT-J 8bit](https://huggingface.co/hivemind/gpt-j-6B-8bit). |
|
|
|
Here's how to run it: [](https://colab.research.google.com/drive/1lMja-CPc0vm5_-gXNXAWU-9c0nom7vZ9) |
|
|
|
This model can be easily loaded using the `GPTJForCausalLM` functionality: |
|
```python |
|
from transformers import GPTJForCausalLM |
|
model = GPTJForCausalLM.from_pretrained("gustavecortal/gpt-j-fr-covid-news") |
|
``` |
|
|
|
Remember, you have to Monkey-Patch the model before loading it (see Colab above). |
|
|
|
## One thousand AI-generated French headlines related to COVID-19 |
|
|
|
How not to be disoriented in a pandemic era when faced with an immense flow of information? [This page](https://gustavecortal.com/project/covid) features one thousand AI-generated French headlines related to COVID-19. |
|
|
|
## fr-boris |
|
|
|
Boris is a 6B parameter autoregressive language model based on the GPT-J architecture and trained using the [mesh-transformer-jax](https://github.com/kingoflolz/mesh-transformer-jax) codebase. |
|
|
|
Boris was trained on around 78B tokens of French text from the [C4](https://huggingface.co/datasets/c4) dataset. |
|
|
|
## Links |
|
|
|
* [Gustave Cortal](https://twitter.com/gustavecortal) |
