davebulaval commited on
Commit
8db06be
1 Parent(s): 2fbbc4c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -33
README.md CHANGED
@@ -1,42 +1,25 @@
1
  ---
2
- license: mit
3
- datasets:
4
- - davebulaval/CSMD
5
- language:
6
- - en
7
- model-index:
8
- - name: MeaningBERT
9
- results:
10
- - task:
11
- type: assesing-meaning-preservation
12
- dataset:
13
- name: davebulaval/CSMD
14
- type: regression
15
- metrics:
16
- - type: r_squared
17
- value: 0.860
18
- - type: pearsonr
19
- value: 0.928
20
- - type: rmse
21
- value: 16.355
22
-
23
- metrics:
24
- - r_squared
25
- - pearsonr
26
- - rmse
27
-
28
- tags:
29
- - text-simplification
30
- - meaning
31
- - assess
32
  ---
33
 
34
  # Here is MeaningBERT
35
 
36
- MeaningBERT is an automatic and trainable model and [metric]() for assessing meaning preservation between sentences. MeaningBERT was
37
  proposed in our
38
  article [MeaningBERT: assessing meaning preservation between sentences](https://www.frontiersin.org/articles/10.3389/frai.2023.1223924/full).
39
- Its goal is to assess meaning preservation between two sentences that correlate highly with human judgments and sanity checks. For more details, refer to our publicly available article.
 
 
 
 
 
40
 
41
  ## Sanity Check
42
 
@@ -62,7 +45,8 @@ for computer floating-point inaccuracy, we round the ratings to the nearest inte
62
 
63
  Our second test evaluates meaning preservation between a source sentence and an unrelated sentence generated by a large
64
  language model.3 The idea is to verify that the metric finds a meaning preservation rating of 0 when given a completely
65
- irrelevant sentence mainly composed of irrelevant words (also known as word soup). Since this test's expected rating is 0, we check that the metric rating is lower or equal to a threshold value X∈[5, 1].
 
66
  Again, to account for computer floating-point inaccuracy, we round the ratings to the nearest integer and do not use a
67
  a threshold value of 0%.
68
 
 
1
  ---
2
+ title: MeaningBERT
3
+ emoji: 🦀
4
+ colorFrom: purple
5
+ colorTo: indigo
6
+ sdk: gradio
7
+ sdk_version: 4.2.0
8
+ app_file: app.py
9
+ pinned: false
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
10
  ---
11
 
12
  # Here is MeaningBERT
13
 
14
+ MeaningBERT is an automatic and trainable metric for assessing meaning preservation between sentences. MeaningBERT was
15
  proposed in our
16
  article [MeaningBERT: assessing meaning preservation between sentences](https://www.frontiersin.org/articles/10.3389/frai.2023.1223924/full).
17
+ Its goal is to assess meaning preservation between two sentences that correlate highly with human judgments and sanity
18
+ checks. For more details, refer to our publicly available article.
19
+
20
+ > This public version of our model uses the best model trained (where in our article, we present the performance results
21
+ > of an average of 10 models) for a more extended period (1000 epochs instead of 250). We have observed later that the
22
+ > model can further reduce dev loss and increase performance.
23
 
24
  ## Sanity Check
25
 
 
45
 
46
  Our second test evaluates meaning preservation between a source sentence and an unrelated sentence generated by a large
47
  language model.3 The idea is to verify that the metric finds a meaning preservation rating of 0 when given a completely
48
+ irrelevant sentence mainly composed of irrelevant words (also known as word soup). Since this test's expected rating is
49
+ 0, we check that the metric rating is lower or equal to a threshold value X∈[5, 1].
50
  Again, to account for computer floating-point inaccuracy, we round the ratings to the nearest integer and do not use a
51
  a threshold value of 0%.
52