JCLS_model_card.md · AntoineBourgois/BookNLP-fr_coreference-resolution_camembert-large

metadata

language: fr
tags:
  - coreference-resolution
  - anaphora-resolution
  - mentions-linking
  - literary-texts
  - camembert
  - literary-texts
  - nested-entities
  - BookNLP-fr
license: apache-2.0
metrics:
  - MUC
  - B3
  - CEAF
  - CoNLL-F1
base_model:
  - almanach/camembert-large

INTRODUCTION:

This model, developed as part of the BookNLP-fr project, is a coreference resolution model built on top of camembert-large embeddings. It is trained to link mentions of the same entity across a text, focusing on literary works in French.

This specific model has been trained to link entities of the following types: PER.

MODEL PERFORMANCES (LOOCV):

Overall Coreference Resolution Performances for non-overlapping windows of different length:

	Window width (tokens)	Document count	Sample count	MUC F1	B3 F1	CEAFe F1	CONLL F1
0	500	5	64	93.49%	86.27%	77.85%	85.87%
1	1,000	5	30	93.68%	81.32%	71.92%	82.31%
2	2,000	5	14	93.98%	76.90%	67.26%	79.38%
3	5,000	3	5	94.83%	68.34%	59.88%	74.35%
4	10,000	2	2	96.16%	62.22%	57.12%	71.84%

Coreference Resolution Performances on the fully annotated sample for each document:

	Token count	Mention count	MUC F1	B3 F1	CEAFe F1	CONLL F1
0	2,554	330	90.24%	65.27%	72.36%	75.96%
1	2,929	386	95.65%	78.21%	64.23%	79.37%
2	5,425	558	90.46%	53.03%	59.52%	67.67%
3	10,982	1,095	97.18%	65.30%	60.49%	74.32%
4	11,902	1,692	95.03%	58.83%	45.59%	66.49%

TRAINING PARAMETERS:

Entities types: PER
Split strategy: Leave-one-out cross-validation (29 files)
Train/Validation split: 0.85 / 0.15
Batch size: 16,000
Initial learning rate: 0.0004
Focal loss gamma: 1
Focal loss alpha: 0.25
Pronoun lookup antecedents: 30
Common and Proper nouns lookup antecedents: 300

MODEL ARCHITECTURE:

Model Input: 2,165 dimensions vector

Concatenated maximum context camembert-large embeddings (2 * 1,024 = 2,048 dimensions)
Additional mentions features (106 dimensions):
- Length of mentions
- Position of the mention's start token within the sentence
- Grammatical category of the mentions (pronoun, common noun, proper noun)
- Dependency relation of the mention's head (one-hot encoded)
- Gender of the mentions (one-hot encoded)
- Number (singular/plural) of the mentions (one-hot encoded)
- Grammatical person of the mentions (one-hot encoded)
Additional mention pairs features (11 dimensions):
- Distance between mention IDs
- Distance between start tokens of mentions
- Distance between end tokens of mentions
- Distance between sentences containing mentions
- Distance between paragraphs containing mentions
- Difference in nesting levels of mentions
- Ratio of shared tokens between mentions
- Exact text match between mentions (binary)
- Exact match of mention heads (binary)
- Match of syntactic heads between mentions (binary)
- Match of entity types between mentions (binary)
Hidden Layers:
- Number of layers: 3
- Units per layer: 1,900 nodes
- Activation function: relu
- Dropout rate: 0.6
Final Layer:
- Type: Linear
- Input: 1900 dimensions
- Output: 1 dimension (mention pair coreference score)

Model Output: Continuous prediction between 0 (not coreferent) and 1 (coreferent) indicating the degree of confidence.

HOW TO USE:

*** IN CONSTRUCTION ***

TRAINING CORPUS:

	Document	Tokens Count	Is included in model eval
0	1836_Gautier-Theophile_La-morte-amoureuse	14,299 tokens	False
1	1840_Sand-George_Pauline	12,315 tokens	False
2	1842_Balzac-Honore-de_La-Maison-du-chat-qui-pelote	24,776 tokens	False
3	1844_Balzac-Honore-de_La-Maison-Nucingen	30,987 tokens	False
4	1844_Balzac-Honore-de_Sarrasine	15,408 tokens	False
5	1856_Cousin-Victor_Madame-de-Hautefort	11,768 tokens	False
6	1863_Gautier-Theophile_Le-capitaine-Fracasse	11,834 tokens	False
7	1873_Zola-Emile_Le-ventre-de-Paris	12,557 tokens	False
8	1881_Flaubert-Gustave_Bouvard-et-Pecuchet	12,281 tokens	False
9	1882_Guy-de-Maupassant_Mademoiselle-Fifi-1_1-MADEMOISELLE-FIFI	5,425 tokens	True
10	1882_Guy-de-Maupassant_Mademoiselle-Fifi-1_2-MADAME-BAPTISTE	2,554 tokens	True
11	1882_Guy-de-Maupassant_Mademoiselle-Fifi-1_3-LA-ROUILLE	2,929 tokens	True
12	1882_Guy-de-Maupassant_Mademoiselle-Fifi-2_1-MARROCA	4,067 tokens	False
13	1882_Guy-de-Maupassant_Mademoiselle-Fifi-2_2-LA-BUCHE	2,251 tokens	False
14	1882_Guy-de-Maupassant_Mademoiselle-Fifi-2_3-LA-RELIQUE	2,034 tokens	False
15	1882_Guy-de-Maupassant_Mademoiselle-Fifi-3_1-FOU	1,864 tokens	False
16	1882_Guy-de-Maupassant_Mademoiselle-Fifi-3_2-REVEIL	2,141 tokens	False
17	1882_Guy-de-Maupassant_Mademoiselle-Fifi-3_3-UNE-RUSE	2,441 tokens	False
18	1882_Guy-de-Maupassant_Mademoiselle-Fifi-3_4-A-CHEVAL	2,860 tokens	False
19	1882_Guy-de-Maupassant_Mademoiselle-Fifi-3_5-UN-REVEILLON	2,343 tokens	False
20	1901_Lucie-Achard_Rosalie-de-Constant-sa-famille-et-ses-amis	12,703 tokens	False
21	1903_Conan-Laure_Elisabeth_Seton	13,023 tokens	False
22	1904_Rolland-Romain_Jean-Christophe_Tome-I-L-aube	10,982 tokens	True
23	1904_Rolland-Romain_Jean-Christophe_Tome-II-Le-matin	10,305 tokens	False
24	1917_Adèle-Bourgeois_Némoville	12,389 tokens	False
25	1923_Radiguet-Raymond_Le-diable-au-corps	14,637 tokens	False
26	1926_Audoux-Marguerite_De-la-ville-au-moulin	11,902 tokens	True
27	1937_Audoux-Marguerite_Douce-Lumiere	12,285 tokens	False
28	Manon_Lescaut_PEDRO	71,219 tokens	False
29	TOTAL	346,579 tokens	5 files used for cross-validation

CONTACT:

mail: antoine [dot] bourgois [at] protonmail [dot] com