Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,4 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# FintoAI-data-YKL
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
Configurations for maintaining the Annif projects with YKL vocabulary used at [Finto AI service](ai.finto.fi/).
|
| 3 |
|
| 4 |
The projects are trained and evaluated using a [DVC (Data Version Control) pipeline](https://dvc.org/doc/start/data-management/data-pipelines) defined in [dvc.yaml](/dvc.yaml).
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc0-1.0
|
| 3 |
+
language:
|
| 4 |
+
- fi
|
| 5 |
+
- sv
|
| 6 |
+
- en
|
| 7 |
+
pipeline_tag: text-classification
|
| 8 |
+
thumbnail: https://raw.githubusercontent.com/NatLibFi/FintoAI/main/ai.finto.fi/static/img/finto-ai-social.png
|
| 9 |
+
tags:
|
| 10 |
+
- glam
|
| 11 |
+
- lam
|
| 12 |
+
- subject indexing
|
| 13 |
+
- annif
|
| 14 |
+
---
|
| 15 |
# FintoAI-data-YKL
|
| 16 |
+
This repository is for the Annif projects with the
|
| 17 |
+
[YKL vocabulary](https://finto.fi/ykl)
|
| 18 |
+
used at the [Finto AI service](https://ai.finto.fi/).
|
| 19 |
+
The current models were published there 2023-12-04.
|
| 20 |
+
The models have been trained on Python 3.8.10 with [Annif](https://annif.org) version 1.0.0.
|
| 21 |
+
See [projects.toml](projects.toml) for the configurations of the models.
|
| 22 |
+
|
| 23 |
+
This repository is mirrored from GitHub to the 🤗 Hugging Face Hub;
|
| 24 |
+
the GitHub repository does not contain the model files, but only the configurations for the projects and the DVC pipeline, see below.
|
| 25 |
+
|
| 26 |
+
The training corpora that are public can be found from the [Annif-corpora repository](https://github.com/NatLibFi/Annif-corpora/).
|
| 27 |
+
|
| 28 |
+
## Models
|
| 29 |
+
The downloadable directories for projects and vocabularies are stored in the
|
| 30 |
+
[`/data`](https://huggingface.co/juhoinkinen/FintoAI-data-YKL/tree/main/data)
|
| 31 |
+
directory of this repository in the 🤗 Hugging Face Hub.
|
| 32 |
+
|
| 33 |
+
## DVC pipeline
|
| 34 |
+
The projects are trained and evaluated using a [DVC (Data Version Control) pipeline](https://dvc.org/doc/start/data-management/data-pipelines) defined in [dvc.yaml](./dvc.yaml).
|
| 35 |
+
|
| 36 |
+
The pipeline takes care of
|
| 37 |
+
|
| 38 |
+
1. installing Annif in a venv,
|
| 39 |
+
2. loading the vocabulary,
|
| 40 |
+
3. training the projects,
|
| 41 |
+
4. evaluating the projects.
|
| 42 |
+
|
| 43 |
+
When the necessary vocabulary and training corpora are in place the pipeline can be run using the command
|
| 44 |
+
|
| 45 |
+
dvc repro
|
| 46 |
+
|
| 47 |
+
For more information about using DVC with Annif projects see the [DVC exercise of Annif tutorial](https://github.com/NatLibFi/Annif-tutorial/blob/master/exercises/OPT_dvc.md).
|
| 48 |
+
|
| 49 |
Configurations for maintaining the Annif projects with YKL vocabulary used at [Finto AI service](ai.finto.fi/).
|
| 50 |
|
| 51 |
The projects are trained and evaluated using a [DVC (Data Version Control) pipeline](https://dvc.org/doc/start/data-management/data-pipelines) defined in [dvc.yaml](/dvc.yaml).
|