Spaces:
Running
Running
Jae-Won Chung
commited on
Commit
•
68300e6
1
Parent(s):
06ce58a
Add two more commits to lm-evaluation-harness
Browse files- docs/leaderboard.md +1 -1
- pegasus/README.md +1 -1
docs/leaderboard.md
CHANGED
@@ -42,7 +42,7 @@ Find our benchmark script for one model [here](https://github.com/ml-energy/lead
|
|
42 |
- PyTorch 2.0.1
|
43 |
- [Zeus](https://ml.energy/zeus) -- For GPU time and energy measurement
|
44 |
- [FastChat](https://github.com/lm-sys/fastchat) -- For running inference on various models
|
45 |
-
- [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/
|
46 |
|
47 |
### Hardware
|
48 |
|
|
|
42 |
- PyTorch 2.0.1
|
43 |
- [Zeus](https://ml.energy/zeus) -- For GPU time and energy measurement
|
44 |
- [FastChat](https://github.com/lm-sys/fastchat) -- For running inference on various models
|
45 |
+
- [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/e9f1af36d2f6f8449e3cd132e6885d3b010ec838) -- For NLP evaluation metrics
|
46 |
|
47 |
### Hardware
|
48 |
|
pegasus/README.md
CHANGED
@@ -65,7 +65,7 @@ After all the tasks finish, aggregate all the data into one node and run [`compu
|
|
65 |
|
66 |
## NLP benchmark
|
67 |
|
68 |
-
We'll use [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/
|
69 |
|
70 |
Use Pegasus to run benchmarks for all the models across all nodes.
|
71 |
|
|
|
65 |
|
66 |
## NLP benchmark
|
67 |
|
68 |
+
We'll use [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/e9f1af36d2f6f8449e3cd132e6885d3b010ec838) to run models through three NLP datasets: ARC challenge (`arc`), HellaSwag (`hellaswag`), and TruthfulQA (`truthfulqa`).
|
69 |
|
70 |
Use Pegasus to run benchmarks for all the models across all nodes.
|
71 |
|