MarcosDib commited on
Commit
d6b86d1
1 Parent(s): e8d1c96

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -8
README.md CHANGED
@@ -58,6 +58,13 @@ https://github.com/Marcosdib/S2Query/Classification_Architecture_model.png
58
 
59
  ## Model variations
60
 
 
 
 
 
 
 
 
61
  XXXX has originally been released in base and large variations, for cased and uncased input text. The uncased models
62
  also strips out an accent markers. Chinese and multilingual uncased and cased versions followed shortly after.
63
  Modified preprocessing with whole word masking has replaced subpiece masking in a following work, with the release of
@@ -65,15 +72,15 @@ two models.
65
 
66
  Other 24 smaller models are released afterward.
67
 
68
- The detailed release history can be found on the [google-research/bert readme](https://www.google.com) on github.
69
 
70
- | Model | #params | Language |
71
- |------------------------|--------------------------------|-------|
72
- | [`mcti-base-uncased`]| 110M | English |
73
- | [`mcti-large-uncased`]| 340M | English | sub
74
- | [`mcti-base-cased`]| | 110M | English |
75
- | [`mcti-large-cased`] | 110M | Chinese |
76
- | [`-base-multilingual-cased`] | 110M | Multiple |
77
 
78
  ## Intended uses
79
 
 
58
 
59
  ## Model variations
60
 
61
+ With the motivation to increase accuracy obtained with baseline implementation, we implemented a transfer learning
62
+ strategy under the assumption that small data available for training was insufficient for adequate embedding training.
63
+ In this context, we considered two approaches:
64
+
65
+ i) pre-training wordembeddings using similar datasets for text classification;
66
+ ii) using transformers and attention mechanisms (Longformer) to create contextualized embeddings.
67
+
68
  XXXX has originally been released in base and large variations, for cased and uncased input text. The uncased models
69
  also strips out an accent markers. Chinese and multilingual uncased and cased versions followed shortly after.
70
  Modified preprocessing with whole word masking has replaced subpiece masking in a following work, with the release of
 
72
 
73
  Other 24 smaller models are released afterward.
74
 
75
+ The detailed release history can be found on the [here](https://huggingface.co/unb-lamfo-nlp-mcti) on github.
76
 
77
+ | Model | #params | Language |
78
+ |------------------------------|--------------------|-------|
79
+ | [`mcti-base-uncased`] | 110M | English |
80
+ | [`mcti-large-uncased`] | 340M | English | sub
81
+ | [`mcti-base-cased`] | 110M | English |
82
+ | [`mcti-large-cased`] | 110M | Chinese |
83
+ | [`-base-multilingual-cased`] | 110M | Multiple |
84
 
85
  ## Intended uses
86