Jae-Won Chung commited on
Commit
68300e6
1 Parent(s): 06ce58a

Add two more commits to lm-evaluation-harness

Browse files
Files changed (2) hide show
  1. docs/leaderboard.md +1 -1
  2. pegasus/README.md +1 -1
docs/leaderboard.md CHANGED
@@ -42,7 +42,7 @@ Find our benchmark script for one model [here](https://github.com/ml-energy/lead
42
  - PyTorch 2.0.1
43
  - [Zeus](https://ml.energy/zeus) -- For GPU time and energy measurement
44
  - [FastChat](https://github.com/lm-sys/fastchat) -- For running inference on various models
45
- - [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/72b7f0c00a6ff94632c5b873fc24e093ae74fa47) -- For NLP evaluation metrics
46
 
47
  ### Hardware
48
 
 
42
  - PyTorch 2.0.1
43
  - [Zeus](https://ml.energy/zeus) -- For GPU time and energy measurement
44
  - [FastChat](https://github.com/lm-sys/fastchat) -- For running inference on various models
45
+ - [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/e9f1af36d2f6f8449e3cd132e6885d3b010ec838) -- For NLP evaluation metrics
46
 
47
  ### Hardware
48
 
pegasus/README.md CHANGED
@@ -65,7 +65,7 @@ After all the tasks finish, aggregate all the data into one node and run [`compu
65
 
66
  ## NLP benchmark
67
 
68
- We'll use [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/72b7f0c00a6ff94632c5b873fc24e093ae74fa47) to run models through three NLP datasets: ARC challenge (`arc`), HellaSwag (`hellaswag`), and TruthfulQA (`truthfulqa`).
69
 
70
  Use Pegasus to run benchmarks for all the models across all nodes.
71
 
 
65
 
66
  ## NLP benchmark
67
 
68
+ We'll use [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness/tree/e9f1af36d2f6f8449e3cd132e6885d3b010ec838) to run models through three NLP datasets: ARC challenge (`arc`), HellaSwag (`hellaswag`), and TruthfulQA (`truthfulqa`).
69
 
70
  Use Pegasus to run benchmarks for all the models across all nodes.
71