---
model-index:
- name: voyage-code-3
  results:
  - dataset:
      config: default
      name: MTEB AppsRetrieval (default)
      revision: f22508f96b7a36c2415181ed8bb76f76e04ae2d5
      split: test
      type: CoIR-Retrieval/apps
    metrics:
    - type: main_score
      value: 93.621
    - type: map_at_1
      value: 87.968
    - type: map_at_10
      value: 92.047
    - type: map_at_100
      value: 92.11500000000001
    - type: map_at_1000
      value: 92.11500000000001
    - type: map_at_20
      value: 92.091
    - type: map_at_3
      value: 91.532
    - type: map_at_5
      value: 91.874
    - type: mrr_at_1
      value: 87.99468791500664
    - type: mrr_at_10
      value: 92.0623432197138
    - type: mrr_at_100
      value: 92.13012638701028
    - type: mrr_at_1000
      value: 92.1308801595124
    - type: mrr_at_20
      value: 92.10639248723756
    - type: mrr_at_3
      value: 91.54050464807443
    - type: mrr_at_5
      value: 91.88977423638782
    - type: nauc_map_at_1000_diff1
      value: 90.15490174664647
    - type: nauc_map_at_1000_max
      value: 67.02866848649028
    - type: nauc_map_at_1000_std
      value: -43.43778333375324
    - type: nauc_map_at_100_diff1
      value: 90.15545196165746
    - type: nauc_map_at_100_max
      value: 67.03097567361978
    - type: nauc_map_at_100_std
      value: -43.43758797549862
    - type: nauc_map_at_10_diff1
      value: 90.14099743587612
    - type: nauc_map_at_10_max
      value: 67.16096459752251
    - type: nauc_map_at_10_std
      value: -43.44370410695505
    - type: nauc_map_at_1_diff1
      value: 90.99585708349436
    - type: nauc_map_at_1_max
      value: 63.57782533912141
    - type: nauc_map_at_1_std
      value: -42.361341394612126
    - type: nauc_map_at_20_diff1
      value: 90.1551408410763
    - type: nauc_map_at_20_max
      value: 67.07765073421193
    - type: nauc_map_at_20_std
      value: -43.41448959858248
    - type: nauc_map_at_3_diff1
      value: 89.8995753062064
    - type: nauc_map_at_3_max
      value: 67.61366242819365
    - type: nauc_map_at_3_std
      value: -44.226914049035386
    - type: nauc_map_at_5_diff1
      value: 90.09003396740809
    - type: nauc_map_at_5_max
      value: 67.4059817584288
    - type: nauc_map_at_5_std
      value: -43.62598115209535
    - type: nauc_mrr_at_1000_diff1
      value: 90.1224401379764
    - type: nauc_mrr_at_1000_max
      value: 67.0844465768374
    - type: nauc_mrr_at_1000_std
      value: -43.634995805553366
    - type: nauc_mrr_at_100_diff1
      value: 90.12299449341405
    - type: nauc_mrr_at_100_max
      value: 67.08675243260438
    - type: nauc_mrr_at_100_std
      value: -43.634780889033514
    - type: nauc_mrr_at_10_diff1
      value: 90.10879634384949
    - type: nauc_mrr_at_10_max
      value: 67.21648611418671
    - type: nauc_mrr_at_10_std
      value: -43.63919542502753
    - type: nauc_mrr_at_1_diff1
      value: 90.94858533614968
    - type: nauc_mrr_at_1_max
      value: 63.65698161247239
    - type: nauc_mrr_at_1_std
      value: -42.61438951155136
    - type: nauc_mrr_at_20_diff1
      value: 90.1227820706247
    - type: nauc_mrr_at_20_max
      value: 67.1333396707349
    - type: nauc_mrr_at_20_std
      value: -43.61103858801164
    - type: nauc_mrr_at_3_diff1
      value: 89.87704691693779
    - type: nauc_mrr_at_3_max
      value: 67.69227631885192
    - type: nauc_mrr_at_3_std
      value: -44.3319988628551
    - type: nauc_mrr_at_5_diff1
      value: 90.05844149558918
    - type: nauc_mrr_at_5_max
      value: 67.46072486731265
    - type: nauc_mrr_at_5_std
      value: -43.81755835911823
    - type: nauc_ndcg_at_1000_diff1
      value: 90.04496527123014
    - type: nauc_ndcg_at_1000_max
      value: 67.60247091688248
    - type: nauc_ndcg_at_1000_std
      value: -43.03623963675031
    - type: nauc_ndcg_at_100_diff1
      value: 90.070471931644
    - type: nauc_ndcg_at_100_max
      value: 67.68039037445462
    - type: nauc_ndcg_at_100_std
      value: -43.036110157737035
    - type: nauc_ndcg_at_10_diff1
      value: 90.0028263117178
    - type: nauc_ndcg_at_10_max
      value: 68.38084649569117
    - type: nauc_ndcg_at_10_std
      value: -42.96245497355038
    - type: nauc_ndcg_at_1_diff1
      value: 90.99585708349436
    - type: nauc_ndcg_at_1_max
      value: 63.57782533912141
    - type: nauc_ndcg_at_1_std
      value: -42.361341394612126
    - type: nauc_ndcg_at_20_diff1
      value: 90.06473833364464
    - type: nauc_ndcg_at_20_max
      value: 68.04393781250494
    - type: nauc_ndcg_at_20_std
      value: -42.754643190221756
    - type: nauc_ndcg_at_3_diff1
      value: 89.47150366489366
    - type: nauc_ndcg_at_3_max
      value: 69.28073683657388
    - type: nauc_ndcg_at_3_std
      value: -44.80650989406889
    - type: nauc_ndcg_at_5_diff1
      value: 89.85493663258895
    - type: nauc_ndcg_at_5_max
      value: 68.97238224506673
    - type: nauc_ndcg_at_5_std
      value: -43.59588098814929
    - type: nauc_precision_at_1000_diff1
      value: 71.5228076434295
    - type: nauc_precision_at_1000_max
      value: 100.0
    - type: nauc_precision_at_1000_std
      value: 95.64334438407597
    - type: nauc_precision_at_100_diff1
      value: 89.05538400916919
    - type: nauc_precision_at_100_max
      value: 93.59380801456066
    - type: nauc_precision_at_100_std
      value: 3.4218682638907
    - type: nauc_precision_at_10_diff1
      value: 88.82693413470759
    - type: nauc_precision_at_10_max
      value: 83.71281148941699
    - type: nauc_precision_at_10_std
      value: -33.65732721164693
    - type: nauc_precision_at_1_diff1
      value: 90.99585708349436
    - type: nauc_precision_at_1_max
      value: 63.57782533912141
    - type: nauc_precision_at_1_std
      value: -42.361341394612126
    - type: nauc_precision_at_20_diff1
      value: 89.60083706628117
    - type: nauc_precision_at_20_max
      value: 85.08806968533703
    - type: nauc_precision_at_20_std
      value: -22.269540063924854
    - type: nauc_precision_at_3_diff1
      value: 87.16551109941072
    - type: nauc_precision_at_3_max
      value: 78.32435110823177
    - type: nauc_precision_at_3_std
      value: -47.825568017450934
    - type: nauc_precision_at_5_diff1
      value: 88.21645334780426
    - type: nauc_precision_at_5_max
      value: 81.21614910200276
    - type: nauc_precision_at_5_std
      value: -42.6577409583649
    - type: nauc_recall_at_1000_diff1
      value: 71.52280764343946
    - type: nauc_recall_at_1000_max
      value: 100.0
    - type: nauc_recall_at_1000_std
      value: 95.64334438409236
    - type: nauc_recall_at_100_diff1
      value: 89.05538400917727
    - type: nauc_recall_at_100_max
      value: 93.59380801455806
    - type: nauc_recall_at_100_std
      value: 3.4218682638950693
    - type: nauc_recall_at_10_diff1
      value: 88.82693413470616
    - type: nauc_recall_at_10_max
      value: 83.71281148941586
    - type: nauc_recall_at_10_std
      value: -33.65732721164498
    - type: nauc_recall_at_1_diff1
      value: 90.99585708349436
    - type: nauc_recall_at_1_max
      value: 63.57782533912141
    - type: nauc_recall_at_1_std
      value: -42.361341394612126
    - type: nauc_recall_at_20_diff1
      value: 89.60083706628052
    - type: nauc_recall_at_20_max
      value: 85.08806968533526
    - type: nauc_recall_at_20_std
      value: -22.269540063922193
    - type: nauc_recall_at_3_diff1
      value: 87.16551109941042
    - type: nauc_recall_at_3_max
      value: 78.32435110823131
    - type: nauc_recall_at_3_std
      value: -47.825568017451694
    - type: nauc_recall_at_5_diff1
      value: 88.21645334780379
    - type: nauc_recall_at_5_max
      value: 81.21614910200307
    - type: nauc_recall_at_5_std
      value: -42.65774095836507
    - type: ndcg_at_1
      value: 87.968
    - type: ndcg_at_10
      value: 93.621
    - type: ndcg_at_100
      value: 93.929
    - type: ndcg_at_1000
      value: 93.949
    - type: ndcg_at_20
      value: 93.77799999999999
    - type: ndcg_at_3
      value: 92.60199999999999
    - type: ndcg_at_5
      value: 93.217
    - type: precision_at_1
      value: 87.968
    - type: precision_at_10
      value: 9.838
    - type: precision_at_100
      value: 0.9979999999999999
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.95
    - type: precision_at_3
      value: 31.89
    - type: precision_at_5
      value: 19.432
    - type: recall_at_1
      value: 87.968
    - type: recall_at_10
      value: 98.38
    - type: recall_at_100
      value: 99.761
    - type: recall_at_1000
      value: 99.92
    - type: recall_at_20
      value: 98.991
    - type: recall_at_3
      value: 95.67099999999999
    - type: recall_at_5
      value: 97.158
    task:
      type: Retrieval
  - dataset:
      config: python
      name: MTEB COIRCodeSearchNetRetrieval (python)
      revision: 4adc7bc41202b5c13543c9c886a25f340634dab3
      split: test
      type: CoIR-Retrieval/CodeSearchNet
    metrics:
    - type: main_score
      value: 98.367
    - type: map_at_1
      value: 96.494
    - type: map_at_10
      value: 97.881
    - type: map_at_100
      value: 97.89099999999999
    - type: map_at_1000
      value: 97.89099999999999
    - type: map_at_20
      value: 97.88799999999999
    - type: map_at_3
      value: 97.761
    - type: map_at_5
      value: 97.849
    - type: mrr_at_1
      value: 96.49416811905081
    - type: mrr_at_10
      value: 97.88111783570305
    - type: mrr_at_100
      value: 97.8906549372564
    - type: mrr_at_1000
      value: 97.89071223051654
    - type: mrr_at_20
      value: 97.88807761475906
    - type: mrr_at_3
      value: 97.76109398042645
    - type: mrr_at_5
      value: 97.84924252580788
    - type: nauc_map_at_1000_diff1
      value: 98.15528913593722
    - type: nauc_map_at_1000_max
      value: 85.10151231357158
    - type: nauc_map_at_1000_std
      value: -42.70593910947264
    - type: nauc_map_at_100_diff1
      value: 98.15533924129993
    - type: nauc_map_at_100_max
      value: 85.10116247232071
    - type: nauc_map_at_100_std
      value: -42.70950485228204
    - type: nauc_map_at_10_diff1
      value: 98.15404628849377
    - type: nauc_map_at_10_max
      value: 85.129420000526
    - type: nauc_map_at_10_std
      value: -43.05569868322259
    - type: nauc_map_at_1_diff1
      value: 98.05311494766504
    - type: nauc_map_at_1_max
      value: 83.37279874457242
    - type: nauc_map_at_1_std
      value: -36.78783832567891
    - type: nauc_map_at_20_diff1
      value: 98.15680755256112
    - type: nauc_map_at_20_max
      value: 85.1144859632837
    - type: nauc_map_at_20_std
      value: -42.76588304453336
    - type: nauc_map_at_3_diff1
      value: 98.19103616317174
    - type: nauc_map_at_3_max
      value: 85.64151924417325
    - type: nauc_map_at_3_std
      value: -43.86751765575472
    - type: nauc_map_at_5_diff1
      value: 98.16782356611095
    - type: nauc_map_at_5_max
      value: 85.28003038549159
    - type: nauc_map_at_5_std
      value: -43.74493490831982
    - type: nauc_mrr_at_1000_diff1
      value: 98.15528913593722
    - type: nauc_mrr_at_1000_max
      value: 85.10151231357158
    - type: nauc_mrr_at_1000_std
      value: -42.70593910947264
    - type: nauc_mrr_at_100_diff1
      value: 98.15533924129993
    - type: nauc_mrr_at_100_max
      value: 85.10116247232071
    - type: nauc_mrr_at_100_std
      value: -42.70950485228204
    - type: nauc_mrr_at_10_diff1
      value: 98.15404628849377
    - type: nauc_mrr_at_10_max
      value: 85.129420000526
    - type: nauc_mrr_at_10_std
      value: -43.05569868322259
    - type: nauc_mrr_at_1_diff1
      value: 98.05311494766504
    - type: nauc_mrr_at_1_max
      value: 83.37279874457242
    - type: nauc_mrr_at_1_std
      value: -36.78783832567891
    - type: nauc_mrr_at_20_diff1
      value: 98.15680755256112
    - type: nauc_mrr_at_20_max
      value: 85.1144859632837
    - type: nauc_mrr_at_20_std
      value: -42.76588304453336
    - type: nauc_mrr_at_3_diff1
      value: 98.19103616317174
    - type: nauc_mrr_at_3_max
      value: 85.64151924417325
    - type: nauc_mrr_at_3_std
      value: -43.86751765575472
    - type: nauc_mrr_at_5_diff1
      value: 98.16782356611095
    - type: nauc_mrr_at_5_max
      value: 85.28003038549159
    - type: nauc_mrr_at_5_std
      value: -43.74493490831982
    - type: nauc_ndcg_at_1000_diff1
      value: 98.16554701617589
    - type: nauc_ndcg_at_1000_max
      value: 85.2748828477838
    - type: nauc_ndcg_at_1000_std
      value: -43.53566701835024
    - type: nauc_ndcg_at_100_diff1
      value: 98.16666937775803
    - type: nauc_ndcg_at_100_max
      value: 85.26689650159106
    - type: nauc_ndcg_at_100_std
      value: -43.61547874575031
    - type: nauc_ndcg_at_10_diff1
      value: 98.15895834975365
    - type: nauc_ndcg_at_10_max
      value: 85.43419484878363
    - type: nauc_ndcg_at_10_std
      value: -45.349653395420944
    - type: nauc_ndcg_at_1_diff1
      value: 98.05311494766504
    - type: nauc_ndcg_at_1_max
      value: 83.37279874457242
    - type: nauc_ndcg_at_1_std
      value: -36.78783832567891
    - type: nauc_ndcg_at_20_diff1
      value: 98.17535928747724
    - type: nauc_ndcg_at_20_max
      value: 85.37092010493639
    - type: nauc_ndcg_at_20_std
      value: -43.98828652303495
    - type: nauc_ndcg_at_3_diff1
      value: 98.25821737148956
    - type: nauc_ndcg_at_3_max
      value: 86.61782450729814
    - type: nauc_ndcg_at_3_std
      value: -47.32915673717569
    - type: nauc_ndcg_at_5_diff1
      value: 98.2061154768608
    - type: nauc_ndcg_at_5_max
      value: 85.87935683953549
    - type: nauc_ndcg_at_5_std
      value: -47.302585930633896
    - type: nauc_precision_at_1000_diff1
      value: .nan
    - type: nauc_precision_at_1000_max
      value: .nan
    - type: nauc_precision_at_1000_std
      value: .nan
    - type: nauc_precision_at_100_diff1
      value: 100.0
    - type: nauc_precision_at_100_max
      value: 72.22153459022752
    - type: nauc_precision_at_100_std
      value: -173.98459168247265
    - type: nauc_precision_at_10_diff1
      value: 97.94083448220364
    - type: nauc_precision_at_10_max
      value: 91.42141149896693
    - type: nauc_precision_at_10_std
      value: -110.27705518270434
    - type: nauc_precision_at_1_diff1
      value: 98.05311494766504
    - type: nauc_precision_at_1_max
      value: 83.37279874457242
    - type: nauc_precision_at_1_std
      value: -36.78783832567891
    - type: nauc_precision_at_20_diff1
      value: 98.99445460386596
    - type: nauc_precision_at_20_max
      value: 93.841013383867
    - type: nauc_precision_at_20_std
      value: -85.33131689356598
    - type: nauc_precision_at_3_diff1
      value: 98.77736667583922
    - type: nauc_precision_at_3_max
      value: 94.04942862307165
    - type: nauc_precision_at_3_std
      value: -74.09082528149943
    - type: nauc_precision_at_5_diff1
      value: 98.69061226878082
    - type: nauc_precision_at_5_max
      value: 93.54052780940046
    - type: nauc_precision_at_5_std
      value: -97.44201029419808
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: 100.0
    - type: nauc_recall_at_100_max
      value: 72.22153459013214
    - type: nauc_recall_at_100_std
      value: -173.98459168226447
    - type: nauc_recall_at_10_diff1
      value: 97.94083448220702
    - type: nauc_recall_at_10_max
      value: 91.42141149896996
    - type: nauc_recall_at_10_std
      value: -110.27705518269966
    - type: nauc_recall_at_1_diff1
      value: 98.05311494766504
    - type: nauc_recall_at_1_max
      value: 83.37279874457242
    - type: nauc_recall_at_1_std
      value: -36.78783832567891
    - type: nauc_recall_at_20_diff1
      value: 98.99445460384722
    - type: nauc_recall_at_20_max
      value: 93.84101338387075
    - type: nauc_recall_at_20_std
      value: -85.33131689356598
    - type: nauc_recall_at_3_diff1
      value: 98.77736667583844
    - type: nauc_recall_at_3_max
      value: 94.0494286230709
    - type: nauc_recall_at_3_std
      value: -74.09082528149965
    - type: nauc_recall_at_5_diff1
      value: 98.69061226878156
    - type: nauc_recall_at_5_max
      value: 93.5405278094034
    - type: nauc_recall_at_5_std
      value: -97.44201029420337
    - type: ndcg_at_1
      value: 96.494
    - type: ndcg_at_10
      value: 98.367
    - type: ndcg_at_100
      value: 98.408
    - type: ndcg_at_1000
      value: 98.409
    - type: ndcg_at_20
      value: 98.392
    - type: ndcg_at_3
      value: 98.13
    - type: ndcg_at_5
      value: 98.289
    - type: precision_at_1
      value: 96.494
    - type: precision_at_10
      value: 9.981
    - type: precision_at_100
      value: 1.0
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.9959999999999996
    - type: precision_at_3
      value: 33.061
    - type: precision_at_5
      value: 19.914
    - type: recall_at_1
      value: 96.494
    - type: recall_at_10
      value: 99.812
    - type: recall_at_100
      value: 99.993
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 99.913
    - type: recall_at_3
      value: 99.182
    - type: recall_at_5
      value: 99.571
    task:
      type: Retrieval
  - dataset:
      config: javascript
      name: MTEB COIRCodeSearchNetRetrieval (javascript)
      revision: 4adc7bc41202b5c13543c9c886a25f340634dab3
      split: test
      type: CoIR-Retrieval/CodeSearchNet
    metrics:
    - type: main_score
      value: 83.479
    - type: map_at_1
      value: 74.354
    - type: map_at_10
      value: 80.71300000000001
    - type: map_at_100
      value: 80.95700000000001
    - type: map_at_1000
      value: 80.963
    - type: map_at_20
      value: 80.87100000000001
    - type: map_at_3
      value: 79.712
    - type: map_at_5
      value: 80.28999999999999
    - type: mrr_at_1
      value: 74.35429960498328
    - type: mrr_at_10
      value: 80.71294728769666
    - type: mrr_at_100
      value: 80.95719685924738
    - type: mrr_at_1000
      value: 80.9626426876391
    - type: mrr_at_20
      value: 80.87121221349531
    - type: mrr_at_3
      value: 79.71234680441609
    - type: mrr_at_5
      value: 80.28967892231339
    - type: nauc_map_at_1000_diff1
      value: 82.88577068822386
    - type: nauc_map_at_1000_max
      value: 67.67862670522469
    - type: nauc_map_at_1000_std
      value: -3.522371246361021
    - type: nauc_map_at_100_diff1
      value: 82.8829326501484
    - type: nauc_map_at_100_max
      value: 67.68203242273655
    - type: nauc_map_at_100_std
      value: -3.522160367469023
    - type: nauc_map_at_10_diff1
      value: 82.8083126411219
    - type: nauc_map_at_10_max
      value: 67.76494610226807
    - type: nauc_map_at_10_std
      value: -3.3188673363705137
    - type: nauc_map_at_1_diff1
      value: 85.06863588556931
    - type: nauc_map_at_1_max
      value: 64.5628789135785
    - type: nauc_map_at_1_std
      value: -6.191657522827031
    - type: nauc_map_at_20_diff1
      value: 82.85819113195612
    - type: nauc_map_at_20_max
      value: 67.75920634659379
    - type: nauc_map_at_20_std
      value: -3.3782092153909744
    - type: nauc_map_at_3_diff1
      value: 82.93140971183675
    - type: nauc_map_at_3_max
      value: 67.88657739487108
    - type: nauc_map_at_3_std
      value: -3.9042280696240383
    - type: nauc_map_at_5_diff1
      value: 82.78238115943125
    - type: nauc_map_at_5_max
      value: 67.78270133394841
    - type: nauc_map_at_5_std
      value: -3.6293201723127693
    - type: nauc_mrr_at_1000_diff1
      value: 82.88577068822386
    - type: nauc_mrr_at_1000_max
      value: 67.67862670522469
    - type: nauc_mrr_at_1000_std
      value: -3.522371246361021
    - type: nauc_mrr_at_100_diff1
      value: 82.8829326501484
    - type: nauc_mrr_at_100_max
      value: 67.68203242273655
    - type: nauc_mrr_at_100_std
      value: -3.522160367469023
    - type: nauc_mrr_at_10_diff1
      value: 82.8083126411219
    - type: nauc_mrr_at_10_max
      value: 67.76494610226807
    - type: nauc_mrr_at_10_std
      value: -3.3188673363705137
    - type: nauc_mrr_at_1_diff1
      value: 85.06863588556931
    - type: nauc_mrr_at_1_max
      value: 64.5628789135785
    - type: nauc_mrr_at_1_std
      value: -6.191657522827031
    - type: nauc_mrr_at_20_diff1
      value: 82.85819113195612
    - type: nauc_mrr_at_20_max
      value: 67.75920634659379
    - type: nauc_mrr_at_20_std
      value: -3.3782092153909744
    - type: nauc_mrr_at_3_diff1
      value: 82.93140971183675
    - type: nauc_mrr_at_3_max
      value: 67.88657739487108
    - type: nauc_mrr_at_3_std
      value: -3.9042280696240383
    - type: nauc_mrr_at_5_diff1
      value: 82.78238115943125
    - type: nauc_mrr_at_5_max
      value: 67.78270133394841
    - type: nauc_mrr_at_5_std
      value: -3.6293201723127693
    - type: nauc_ndcg_at_1000_diff1
      value: 82.3199098700142
    - type: nauc_ndcg_at_1000_max
      value: 68.120342472938
    - type: nauc_ndcg_at_1000_std
      value: -2.8508509319395734
    - type: nauc_ndcg_at_100_diff1
      value: 82.1633159731435
    - type: nauc_ndcg_at_100_max
      value: 68.26748139014519
    - type: nauc_ndcg_at_100_std
      value: -2.714633476731751
    - type: nauc_ndcg_at_10_diff1
      value: 81.82903994006058
    - type: nauc_ndcg_at_10_max
      value: 68.89584312261682
    - type: nauc_ndcg_at_10_std
      value: -1.4252080306379409
    - type: nauc_ndcg_at_1_diff1
      value: 85.06863588556931
    - type: nauc_ndcg_at_1_max
      value: 64.5628789135785
    - type: nauc_ndcg_at_1_std
      value: -6.191657522827031
    - type: nauc_ndcg_at_20_diff1
      value: 81.98789282274657
    - type: nauc_ndcg_at_20_max
      value: 68.8959043317764
    - type: nauc_ndcg_at_20_std
      value: -1.6185846906781556
    - type: nauc_ndcg_at_3_diff1
      value: 82.10544479985309
    - type: nauc_ndcg_at_3_max
      value: 68.98284750281354
    - type: nauc_ndcg_at_3_std
      value: -3.02774137516056
    - type: nauc_ndcg_at_5_diff1
      value: 81.77317252542665
    - type: nauc_ndcg_at_5_max
      value: 68.8430670861106
    - type: nauc_ndcg_at_5_std
      value: -2.3676013953659583
    - type: nauc_precision_at_1000_diff1
      value: 71.2357013205843
    - type: nauc_precision_at_1000_max
      value: 59.54259563274979
    - type: nauc_precision_at_1000_std
      value: -6.453769813732959
    - type: nauc_precision_at_100_diff1
      value: 70.6646143169478
    - type: nauc_precision_at_100_max
      value: 72.21908402194197
    - type: nauc_precision_at_100_std
      value: 3.6390309142884423
    - type: nauc_precision_at_10_diff1
      value: 75.69911710504576
    - type: nauc_precision_at_10_max
      value: 75.9705779221398
    - type: nauc_precision_at_10_std
      value: 11.632934387302264
    - type: nauc_precision_at_1_diff1
      value: 85.06863588556931
    - type: nauc_precision_at_1_max
      value: 64.5628789135785
    - type: nauc_precision_at_1_std
      value: -6.191657522827031
    - type: nauc_precision_at_20_diff1
      value: 74.91685968954766
    - type: nauc_precision_at_20_max
      value: 78.48940148220586
    - type: nauc_precision_at_20_std
      value: 14.477347988107262
    - type: nauc_precision_at_3_diff1
      value: 78.78186792721571
    - type: nauc_precision_at_3_max
      value: 73.31020934353046
    - type: nauc_precision_at_3_std
      value: 0.4957741917380896
    - type: nauc_precision_at_5_diff1
      value: 76.96340948709872
    - type: nauc_precision_at_5_max
      value: 73.6957286199771
    - type: nauc_precision_at_5_std
      value: 3.7822933962717555
    - type: nauc_recall_at_1000_diff1
      value: 71.23570132058367
    - type: nauc_recall_at_1000_max
      value: 59.54259563275034
    - type: nauc_recall_at_1000_std
      value: -6.453769813732673
    - type: nauc_recall_at_100_diff1
      value: 70.66461431694707
    - type: nauc_recall_at_100_max
      value: 72.21908402194222
    - type: nauc_recall_at_100_std
      value: 3.639030914287266
    - type: nauc_recall_at_10_diff1
      value: 75.69911710504581
    - type: nauc_recall_at_10_max
      value: 75.97057792213957
    - type: nauc_recall_at_10_std
      value: 11.632934387302335
    - type: nauc_recall_at_1_diff1
      value: 85.06863588556931
    - type: nauc_recall_at_1_max
      value: 64.5628789135785
    - type: nauc_recall_at_1_std
      value: -6.191657522827031
    - type: nauc_recall_at_20_diff1
      value: 74.91685968954776
    - type: nauc_recall_at_20_max
      value: 78.4894014822059
    - type: nauc_recall_at_20_std
      value: 14.477347988107574
    - type: nauc_recall_at_3_diff1
      value: 78.78186792721584
    - type: nauc_recall_at_3_max
      value: 73.31020934353042
    - type: nauc_recall_at_3_std
      value: 0.49577419173804144
    - type: nauc_recall_at_5_diff1
      value: 76.96340948709855
    - type: nauc_recall_at_5_max
      value: 73.695728619977
    - type: nauc_recall_at_5_std
      value: 3.782293396271726
    - type: ndcg_at_1
      value: 74.354
    - type: ndcg_at_10
      value: 83.479
    - type: ndcg_at_100
      value: 84.65599999999999
    - type: ndcg_at_1000
      value: 84.851
    - type: ndcg_at_20
      value: 84.03500000000001
    - type: ndcg_at_3
      value: 81.42200000000001
    - type: ndcg_at_5
      value: 82.461
    - type: precision_at_1
      value: 74.354
    - type: precision_at_10
      value: 9.200999999999999
    - type: precision_at_100
      value: 0.975
    - type: precision_at_1000
      value: 0.099
    - type: precision_at_20
      value: 4.707999999999999
    - type: precision_at_3
      value: 28.786
    - type: precision_at_5
      value: 17.776
    - type: recall_at_1
      value: 74.354
    - type: recall_at_10
      value: 92.009
    - type: recall_at_100
      value: 97.508
    - type: recall_at_1000
      value: 99.149
    - type: recall_at_20
      value: 94.16600000000001
    - type: recall_at_3
      value: 86.357
    - type: recall_at_5
      value: 88.87899999999999
    task:
      type: Retrieval
  - dataset:
      config: go
      name: MTEB COIRCodeSearchNetRetrieval (go)
      revision: 4adc7bc41202b5c13543c9c886a25f340634dab3
      split: test
      type: CoIR-Retrieval/CodeSearchNet
    metrics:
    - type: main_score
      value: 91.093
    - type: map_at_1
      value: 83.95700000000001
    - type: map_at_10
      value: 89.095
    - type: map_at_100
      value: 89.21900000000001
    - type: map_at_1000
      value: 89.22099999999999
    - type: map_at_20
      value: 89.181
    - type: map_at_3
      value: 88.472
    - type: map_at_5
      value: 88.866
    - type: mrr_at_1
      value: 83.95715341049002
    - type: mrr_at_10
      value: 89.0947231309824
    - type: mrr_at_100
      value: 89.21908275588152
    - type: mrr_at_1000
      value: 89.22056604897018
    - type: mrr_at_20
      value: 89.18054785129605
    - type: mrr_at_3
      value: 88.47164081096584
    - type: mrr_at_5
      value: 88.86624805056191
    - type: nauc_map_at_1000_diff1
      value: 86.24973579936615
    - type: nauc_map_at_1000_max
      value: 43.76713186644932
    - type: nauc_map_at_1000_std
      value: -47.20109532688898
    - type: nauc_map_at_100_diff1
      value: 86.24862049034168
    - type: nauc_map_at_100_max
      value: 43.76881022585186
    - type: nauc_map_at_100_std
      value: -47.20426521434973
    - type: nauc_map_at_10_diff1
      value: 86.25350003349301
    - type: nauc_map_at_10_max
      value: 43.771945888748306
    - type: nauc_map_at_10_std
      value: -47.62265974147625
    - type: nauc_map_at_1_diff1
      value: 86.98798340919
    - type: nauc_map_at_1_max
      value: 43.024043355768995
    - type: nauc_map_at_1_std
      value: -41.469271817341955
    - type: nauc_map_at_20_diff1
      value: 86.25325068130977
    - type: nauc_map_at_20_max
      value: 43.80727376815637
    - type: nauc_map_at_20_std
      value: -47.31520043996017
    - type: nauc_map_at_3_diff1
      value: 86.10288537323795
    - type: nauc_map_at_3_max
      value: 43.79879378738717
    - type: nauc_map_at_3_std
      value: -48.546995258168884
    - type: nauc_map_at_5_diff1
      value: 86.18610042665851
    - type: nauc_map_at_5_max
      value: 43.4814674201994
    - type: nauc_map_at_5_std
      value: -48.409761968549084
    - type: nauc_mrr_at_1000_diff1
      value: 86.24973579936615
    - type: nauc_mrr_at_1000_max
      value: 43.76713186644932
    - type: nauc_mrr_at_1000_std
      value: -47.20109532688898
    - type: nauc_mrr_at_100_diff1
      value: 86.24862049034168
    - type: nauc_mrr_at_100_max
      value: 43.76881022585186
    - type: nauc_mrr_at_100_std
      value: -47.20426521434973
    - type: nauc_mrr_at_10_diff1
      value: 86.25350003349301
    - type: nauc_mrr_at_10_max
      value: 43.771945888748306
    - type: nauc_mrr_at_10_std
      value: -47.62265974147625
    - type: nauc_mrr_at_1_diff1
      value: 86.98798340919
    - type: nauc_mrr_at_1_max
      value: 43.024043355768995
    - type: nauc_mrr_at_1_std
      value: -41.469271817341955
    - type: nauc_mrr_at_20_diff1
      value: 86.25325068130977
    - type: nauc_mrr_at_20_max
      value: 43.80727376815637
    - type: nauc_mrr_at_20_std
      value: -47.31520043996017
    - type: nauc_mrr_at_3_diff1
      value: 86.10288537323795
    - type: nauc_mrr_at_3_max
      value: 43.79879378738717
    - type: nauc_mrr_at_3_std
      value: -48.546995258168884
    - type: nauc_mrr_at_5_diff1
      value: 86.18610042665851
    - type: nauc_mrr_at_5_max
      value: 43.4814674201994
    - type: nauc_mrr_at_5_std
      value: -48.409761968549084
    - type: nauc_ndcg_at_1000_diff1
      value: 86.1495231100838
    - type: nauc_ndcg_at_1000_max
      value: 44.07212654503798
    - type: nauc_ndcg_at_1000_std
      value: -47.62163979276151
    - type: nauc_ndcg_at_100_diff1
      value: 86.0998353963385
    - type: nauc_ndcg_at_100_max
      value: 44.12175911440964
    - type: nauc_ndcg_at_100_std
      value: -47.60497182589526
    - type: nauc_ndcg_at_10_diff1
      value: 86.15470113116336
    - type: nauc_ndcg_at_10_max
      value: 44.22094281301947
    - type: nauc_ndcg_at_10_std
      value: -49.68035065822525
    - type: nauc_ndcg_at_1_diff1
      value: 86.98798340919
    - type: nauc_ndcg_at_1_max
      value: 43.024043355768995
    - type: nauc_ndcg_at_1_std
      value: -41.469271817341955
    - type: nauc_ndcg_at_20_diff1
      value: 86.153687239383
    - type: nauc_ndcg_at_20_max
      value: 44.413683910958994
    - type: nauc_ndcg_at_20_std
      value: -48.390979780431394
    - type: nauc_ndcg_at_3_diff1
      value: 85.81806042008513
    - type: nauc_ndcg_at_3_max
      value: 44.05606453052025
    - type: nauc_ndcg_at_3_std
      value: -51.77857306868798
    - type: nauc_ndcg_at_5_diff1
      value: 85.97978206190466
    - type: nauc_ndcg_at_5_max
      value: 43.41162519222535
    - type: nauc_ndcg_at_5_std
      value: -51.75858549128195
    - type: nauc_precision_at_1000_diff1
      value: 80.08890508034311
    - type: nauc_precision_at_1000_max
      value: 94.23440421215071
    - type: nauc_precision_at_1000_std
      value: 55.93973044712788
    - type: nauc_precision_at_100_diff1
      value: 75.93187211974605
    - type: nauc_precision_at_100_max
      value: 67.07999533740335
    - type: nauc_precision_at_100_std
      value: -8.479533178162365
    - type: nauc_precision_at_10_diff1
      value: 85.56280199155816
    - type: nauc_precision_at_10_max
      value: 49.46503821359178
    - type: nauc_precision_at_10_std
      value: -68.7017504439551
    - type: nauc_precision_at_1_diff1
      value: 86.98798340919
    - type: nauc_precision_at_1_max
      value: 43.024043355768995
    - type: nauc_precision_at_1_std
      value: -41.469271817341955
    - type: nauc_precision_at_20_diff1
      value: 85.16456431737308
    - type: nauc_precision_at_20_max
      value: 57.105231459014604
    - type: nauc_precision_at_20_std
      value: -57.20616563582712
    - type: nauc_precision_at_3_diff1
      value: 84.42151445378559
    - type: nauc_precision_at_3_max
      value: 45.32354792405939
    - type: nauc_precision_at_3_std
      value: -68.34786794101257
    - type: nauc_precision_at_5_diff1
      value: 84.72626746318639
    - type: nauc_precision_at_5_max
      value: 42.65450965313161
    - type: nauc_precision_at_5_std
      value: -74.63108495140406
    - type: nauc_recall_at_1000_diff1
      value: 80.08890508033205
    - type: nauc_recall_at_1000_max
      value: 94.23440421215781
    - type: nauc_recall_at_1000_std
      value: 55.93973044711981
    - type: nauc_recall_at_100_diff1
      value: 75.93187211975196
    - type: nauc_recall_at_100_max
      value: 67.07999533740575
    - type: nauc_recall_at_100_std
      value: -8.479533178153222
    - type: nauc_recall_at_10_diff1
      value: 85.56280199155815
    - type: nauc_recall_at_10_max
      value: 49.46503821359128
    - type: nauc_recall_at_10_std
      value: -68.70175044395225
    - type: nauc_recall_at_1_diff1
      value: 86.98798340919
    - type: nauc_recall_at_1_max
      value: 43.024043355768995
    - type: nauc_recall_at_1_std
      value: -41.469271817341955
    - type: nauc_recall_at_20_diff1
      value: 85.16456431737305
    - type: nauc_recall_at_20_max
      value: 57.10523145901444
    - type: nauc_recall_at_20_std
      value: -57.20616563582539
    - type: nauc_recall_at_3_diff1
      value: 84.42151445378548
    - type: nauc_recall_at_3_max
      value: 45.32354792405929
    - type: nauc_recall_at_3_std
      value: -68.34786794101288
    - type: nauc_recall_at_5_diff1
      value: 84.72626746318646
    - type: nauc_recall_at_5_max
      value: 42.65450965313161
    - type: nauc_recall_at_5_std
      value: -74.63108495140318
    - type: ndcg_at_1
      value: 83.95700000000001
    - type: ndcg_at_10
      value: 91.093
    - type: ndcg_at_100
      value: 91.644
    - type: ndcg_at_1000
      value: 91.67999999999999
    - type: ndcg_at_20
      value: 91.39699999999999
    - type: ndcg_at_3
      value: 89.84299999999999
    - type: ndcg_at_5
      value: 90.551
    - type: precision_at_1
      value: 83.95700000000001
    - type: precision_at_10
      value: 9.714
    - type: precision_at_100
      value: 0.996
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.916
    - type: precision_at_3
      value: 31.261
    - type: precision_at_5
      value: 19.099
    - type: recall_at_1
      value: 83.95700000000001
    - type: recall_at_10
      value: 97.14399999999999
    - type: recall_at_100
      value: 99.59400000000001
    - type: recall_at_1000
      value: 99.87700000000001
    - type: recall_at_20
      value: 98.32600000000001
    - type: recall_at_3
      value: 93.782
    - type: recall_at_5
      value: 95.494
    task:
      type: Retrieval
  - dataset:
      config: ruby
      name: MTEB COIRCodeSearchNetRetrieval (ruby)
      revision: 4adc7bc41202b5c13543c9c886a25f340634dab3
      split: test
      type: CoIR-Retrieval/CodeSearchNet
    metrics:
    - type: main_score
      value: 86.521
    - type: map_at_1
      value: 77.082
    - type: map_at_10
      value: 83.552
    - type: map_at_100
      value: 83.687
    - type: map_at_1000
      value: 83.69
    - type: map_at_20
      value: 83.64200000000001
    - type: map_at_3
      value: 82.11699999999999
    - type: map_at_5
      value: 83.164
    - type: mrr_at_1
      value: 77.08168120539256
    - type: mrr_at_10
      value: 83.55194919627917
    - type: mrr_at_100
      value: 83.68701866558312
    - type: mrr_at_1000
      value: 83.6903032381438
    - type: mrr_at_20
      value: 83.64219478925247
    - type: mrr_at_3
      value: 82.11736716891363
    - type: mrr_at_5
      value: 83.16415543219676
    - type: nauc_map_at_1000_diff1
      value: 79.64679377977846
    - type: nauc_map_at_1000_max
      value: 70.74499431961593
    - type: nauc_map_at_1000_std
      value: -14.853409296691982
    - type: nauc_map_at_100_diff1
      value: 79.64499589721636
    - type: nauc_map_at_100_max
      value: 70.74759843854231
    - type: nauc_map_at_100_std
      value: -14.844258734825894
    - type: nauc_map_at_10_diff1
      value: 79.53290732095073
    - type: nauc_map_at_10_max
      value: 70.76525352436302
    - type: nauc_map_at_10_std
      value: -14.744825626716187
    - type: nauc_map_at_1_diff1
      value: 82.27728044611361
    - type: nauc_map_at_1_max
      value: 69.29382107125889
    - type: nauc_map_at_1_std
      value: -16.02966665531533
    - type: nauc_map_at_20_diff1
      value: 79.57275942354842
    - type: nauc_map_at_20_max
      value: 70.73838311952618
    - type: nauc_map_at_20_std
      value: -14.86925109860625
    - type: nauc_map_at_3_diff1
      value: 79.47572156907859
    - type: nauc_map_at_3_max
      value: 70.85276903670264
    - type: nauc_map_at_3_std
      value: -15.755792005612602
    - type: nauc_map_at_5_diff1
      value: 79.42518354242955
    - type: nauc_map_at_5_max
      value: 70.69226154953026
    - type: nauc_map_at_5_std
      value: -15.473047773228993
    - type: nauc_mrr_at_1000_diff1
      value: 79.64679377977846
    - type: nauc_mrr_at_1000_max
      value: 70.74499431961593
    - type: nauc_mrr_at_1000_std
      value: -14.853409296691982
    - type: nauc_mrr_at_100_diff1
      value: 79.64499589721636
    - type: nauc_mrr_at_100_max
      value: 70.74759843854231
    - type: nauc_mrr_at_100_std
      value: -14.844258734825894
    - type: nauc_mrr_at_10_diff1
      value: 79.53290732095073
    - type: nauc_mrr_at_10_max
      value: 70.76525352436302
    - type: nauc_mrr_at_10_std
      value: -14.744825626716187
    - type: nauc_mrr_at_1_diff1
      value: 82.27728044611361
    - type: nauc_mrr_at_1_max
      value: 69.29382107125889
    - type: nauc_mrr_at_1_std
      value: -16.02966665531533
    - type: nauc_mrr_at_20_diff1
      value: 79.57275942354842
    - type: nauc_mrr_at_20_max
      value: 70.73838311952618
    - type: nauc_mrr_at_20_std
      value: -14.86925109860625
    - type: nauc_mrr_at_3_diff1
      value: 79.47572156907859
    - type: nauc_mrr_at_3_max
      value: 70.85276903670264
    - type: nauc_mrr_at_3_std
      value: -15.755792005612602
    - type: nauc_mrr_at_5_diff1
      value: 79.42518354242955
    - type: nauc_mrr_at_5_max
      value: 70.69226154953026
    - type: nauc_mrr_at_5_std
      value: -15.473047773228993
    - type: nauc_ndcg_at_1000_diff1
      value: 79.15083604230807
    - type: nauc_ndcg_at_1000_max
      value: 71.1178743998364
    - type: nauc_ndcg_at_1000_std
      value: -14.28015599243118
    - type: nauc_ndcg_at_100_diff1
      value: 79.03711984795275
    - type: nauc_ndcg_at_100_max
      value: 71.20364489826714
    - type: nauc_ndcg_at_100_std
      value: -13.915390197654142
    - type: nauc_ndcg_at_10_diff1
      value: 78.38949093885134
    - type: nauc_ndcg_at_10_max
      value: 71.26426621928836
    - type: nauc_ndcg_at_10_std
      value: -13.497441432215215
    - type: nauc_ndcg_at_1_diff1
      value: 82.27728044611361
    - type: nauc_ndcg_at_1_max
      value: 69.29382107125889
    - type: nauc_ndcg_at_1_std
      value: -16.02966665531533
    - type: nauc_ndcg_at_20_diff1
      value: 78.51349613885007
    - type: nauc_ndcg_at_20_max
      value: 71.13292440385801
    - type: nauc_ndcg_at_20_std
      value: -14.035580557966087
    - type: nauc_ndcg_at_3_diff1
      value: 78.3820255722769
    - type: nauc_ndcg_at_3_max
      value: 71.32668614112852
    - type: nauc_ndcg_at_3_std
      value: -16.04098840431829
    - type: nauc_ndcg_at_5_diff1
      value: 78.14070256474213
    - type: nauc_ndcg_at_5_max
      value: 71.04026107145508
    - type: nauc_ndcg_at_5_std
      value: -15.592128711464067
    - type: nauc_precision_at_1000_diff1
      value: 77.2067992384371
    - type: nauc_precision_at_1000_max
      value: 85.55623308018635
    - type: nauc_precision_at_1000_std
      value: -3.0163245425889005
    - type: nauc_precision_at_100_diff1
      value: 71.01994754486307
    - type: nauc_precision_at_100_max
      value: 82.30803790261785
    - type: nauc_precision_at_100_std
      value: 12.633506693103701
    - type: nauc_precision_at_10_diff1
      value: 67.04993913892164
    - type: nauc_precision_at_10_max
      value: 76.23886937696508
    - type: nauc_precision_at_10_std
      value: 1.5250700897226326
    - type: nauc_precision_at_1_diff1
      value: 82.27728044611361
    - type: nauc_precision_at_1_max
      value: 69.29382107125889
    - type: nauc_precision_at_1_std
      value: -16.02966665531533
    - type: nauc_precision_at_20_diff1
      value: 63.666986838468034
    - type: nauc_precision_at_20_max
      value: 76.14090190912508
    - type: nauc_precision_at_20_std
      value: -0.5871594787522816
    - type: nauc_precision_at_3_diff1
      value: 73.86799722582569
    - type: nauc_precision_at_3_max
      value: 73.220598048437
    - type: nauc_precision_at_3_std
      value: -17.398868434567206
    - type: nauc_precision_at_5_diff1
      value: 70.02914493269556
    - type: nauc_precision_at_5_max
      value: 73.00069716131422
    - type: nauc_precision_at_5_std
      value: -16.614393845477327
    - type: nauc_recall_at_1000_diff1
      value: 77.20679923843755
    - type: nauc_recall_at_1000_max
      value: 85.55623308018376
    - type: nauc_recall_at_1000_std
      value: -3.016324542591852
    - type: nauc_recall_at_100_diff1
      value: 71.01994754486401
    - type: nauc_recall_at_100_max
      value: 82.30803790261768
    - type: nauc_recall_at_100_std
      value: 12.633506693104875
    - type: nauc_recall_at_10_diff1
      value: 67.04993913892174
    - type: nauc_recall_at_10_max
      value: 76.23886937696521
    - type: nauc_recall_at_10_std
      value: 1.525070089722257
    - type: nauc_recall_at_1_diff1
      value: 82.27728044611361
    - type: nauc_recall_at_1_max
      value: 69.29382107125889
    - type: nauc_recall_at_1_std
      value: -16.02966665531533
    - type: nauc_recall_at_20_diff1
      value: 63.66698683846833
    - type: nauc_recall_at_20_max
      value: 76.1409019091247
    - type: nauc_recall_at_20_std
      value: -0.5871594787518352
    - type: nauc_recall_at_3_diff1
      value: 73.86799722582569
    - type: nauc_recall_at_3_max
      value: 73.22059804843683
    - type: nauc_recall_at_3_std
      value: -17.398868434566936
    - type: nauc_recall_at_5_diff1
      value: 70.02914493269549
    - type: nauc_recall_at_5_max
      value: 73.00069716131398
    - type: nauc_recall_at_5_std
      value: -16.614393845477526
    - type: ndcg_at_1
      value: 77.082
    - type: ndcg_at_10
      value: 86.521
    - type: ndcg_at_100
      value: 87.117
    - type: ndcg_at_1000
      value: 87.23100000000001
    - type: ndcg_at_20
      value: 86.858
    - type: ndcg_at_3
      value: 83.7
    - type: ndcg_at_5
      value: 85.591
    - type: precision_at_1
      value: 77.082
    - type: precision_at_10
      value: 9.572
    - type: precision_at_100
      value: 0.983
    - type: precision_at_1000
      value: 0.099
    - type: precision_at_20
      value: 4.853
    - type: precision_at_3
      value: 29.421000000000003
    - type: precision_at_5
      value: 18.573
    - type: recall_at_1
      value: 77.082
    - type: recall_at_10
      value: 95.718
    - type: recall_at_100
      value: 98.335
    - type: recall_at_1000
      value: 99.286
    - type: recall_at_20
      value: 97.066
    - type: recall_at_3
      value: 88.263
    - type: recall_at_5
      value: 92.863
    task:
      type: Retrieval
  - dataset:
      config: java
      name: MTEB COIRCodeSearchNetRetrieval (java)
      revision: 4adc7bc41202b5c13543c9c886a25f340634dab3
      split: test
      type: CoIR-Retrieval/CodeSearchNet
    metrics:
    - type: main_score
      value: 87.26899999999999
    - type: map_at_1
      value: 78.84100000000001
    - type: map_at_10
      value: 84.821
    - type: map_at_100
      value: 84.966
    - type: map_at_1000
      value: 84.97200000000001
    - type: map_at_20
      value: 84.91199999999999
    - type: map_at_3
      value: 83.949
    - type: map_at_5
      value: 84.514
    - type: mrr_at_1
      value: 78.8407120036513
    - type: mrr_at_10
      value: 84.82137604775663
    - type: mrr_at_100
      value: 84.96600236904305
    - type: mrr_at_1000
      value: 84.97188010768014
    - type: mrr_at_20
      value: 84.91215021114384
    - type: mrr_at_3
      value: 83.94949033926656
    - type: mrr_at_5
      value: 84.51361630914327
    - type: nauc_map_at_1000_diff1
      value: 81.29745727169212
    - type: nauc_map_at_1000_max
      value: 61.3738482977934
    - type: nauc_map_at_1000_std
      value: -14.611667076737653
    - type: nauc_map_at_100_diff1
      value: 81.29611575498951
    - type: nauc_map_at_100_max
      value: 61.38415642876098
    - type: nauc_map_at_100_std
      value: -14.60020471629818
    - type: nauc_map_at_10_diff1
      value: 81.24492887826995
    - type: nauc_map_at_10_max
      value: 61.476804728960545
    - type: nauc_map_at_10_std
      value: -14.7216428942423
    - type: nauc_map_at_1_diff1
      value: 83.07021879054804
    - type: nauc_map_at_1_max
      value: 57.45256621613931
    - type: nauc_map_at_1_std
      value: -16.94155323782564
    - type: nauc_map_at_20_diff1
      value: 81.28505757844012
    - type: nauc_map_at_20_max
      value: 61.43691967628063
    - type: nauc_map_at_20_std
      value: -14.583239037636378
    - type: nauc_map_at_3_diff1
      value: 81.0101519179173
    - type: nauc_map_at_3_max
      value: 61.45133191553263
    - type: nauc_map_at_3_std
      value: -15.098090101575302
    - type: nauc_map_at_5_diff1
      value: 81.16287036289346
    - type: nauc_map_at_5_max
      value: 61.59538703827228
    - type: nauc_map_at_5_std
      value: -14.843988367319087
    - type: nauc_mrr_at_1000_diff1
      value: 81.29745727169212
    - type: nauc_mrr_at_1000_max
      value: 61.3738482977934
    - type: nauc_mrr_at_1000_std
      value: -14.611667076737653
    - type: nauc_mrr_at_100_diff1
      value: 81.29611575498951
    - type: nauc_mrr_at_100_max
      value: 61.38415642876098
    - type: nauc_mrr_at_100_std
      value: -14.60020471629818
    - type: nauc_mrr_at_10_diff1
      value: 81.24492887826995
    - type: nauc_mrr_at_10_max
      value: 61.476804728960545
    - type: nauc_mrr_at_10_std
      value: -14.7216428942423
    - type: nauc_mrr_at_1_diff1
      value: 83.07021879054804
    - type: nauc_mrr_at_1_max
      value: 57.45256621613931
    - type: nauc_mrr_at_1_std
      value: -16.94155323782564
    - type: nauc_mrr_at_20_diff1
      value: 81.28505757844012
    - type: nauc_mrr_at_20_max
      value: 61.43691967628063
    - type: nauc_mrr_at_20_std
      value: -14.583239037636378
    - type: nauc_mrr_at_3_diff1
      value: 81.0101519179173
    - type: nauc_mrr_at_3_max
      value: 61.45133191553263
    - type: nauc_mrr_at_3_std
      value: -15.098090101575302
    - type: nauc_mrr_at_5_diff1
      value: 81.16287036289346
    - type: nauc_mrr_at_5_max
      value: 61.59538703827228
    - type: nauc_mrr_at_5_std
      value: -14.843988367319087
    - type: nauc_ndcg_at_1000_diff1
      value: 80.93566924723379
    - type: nauc_ndcg_at_1000_max
      value: 62.39699094971305
    - type: nauc_ndcg_at_1000_std
      value: -13.127328464119836
    - type: nauc_ndcg_at_100_diff1
      value: 80.87553513050011
    - type: nauc_ndcg_at_100_max
      value: 62.72180161053774
    - type: nauc_ndcg_at_100_std
      value: -12.649445624854962
    - type: nauc_ndcg_at_10_diff1
      value: 80.608466242348
    - type: nauc_ndcg_at_10_max
      value: 63.24038696080356
    - type: nauc_ndcg_at_10_std
      value: -13.10528733578738
    - type: nauc_ndcg_at_1_diff1
      value: 83.07021879054804
    - type: nauc_ndcg_at_1_max
      value: 57.45256621613931
    - type: nauc_ndcg_at_1_std
      value: -16.94155323782564
    - type: nauc_ndcg_at_20_diff1
      value: 80.77174027485287
    - type: nauc_ndcg_at_20_max
      value: 63.1297458573586
    - type: nauc_ndcg_at_20_std
      value: -12.43612330644508
    - type: nauc_ndcg_at_3_diff1
      value: 80.1778653998912
    - type: nauc_ndcg_at_3_max
      value: 63.029112839728796
    - type: nauc_ndcg_at_3_std
      value: -14.201414014064817
    - type: nauc_ndcg_at_5_diff1
      value: 80.41507067529044
    - type: nauc_ndcg_at_5_max
      value: 63.456703437014475
    - type: nauc_ndcg_at_5_std
      value: -13.566365227692753
    - type: nauc_precision_at_1000_diff1
      value: 73.89525267370539
    - type: nauc_precision_at_1000_max
      value: 83.46793489687887
    - type: nauc_precision_at_1000_std
      value: 43.36957143495421
    - type: nauc_precision_at_100_diff1
      value: 75.33082548811534
    - type: nauc_precision_at_100_max
      value: 85.08356966408422
    - type: nauc_precision_at_100_std
      value: 30.91532174961418
    - type: nauc_precision_at_10_diff1
      value: 75.96120115397336
    - type: nauc_precision_at_10_max
      value: 76.87957072290979
    - type: nauc_precision_at_10_std
      value: 0.48296998647622513
    - type: nauc_precision_at_1_diff1
      value: 83.07021879054804
    - type: nauc_precision_at_1_max
      value: 57.45256621613931
    - type: nauc_precision_at_1_std
      value: -16.94155323782564
    - type: nauc_precision_at_20_diff1
      value: 76.37431221666498
    - type: nauc_precision_at_20_max
      value: 80.17866322470198
    - type: nauc_precision_at_20_std
      value: 12.914278225658554
    - type: nauc_precision_at_3_diff1
      value: 76.55984367763438
    - type: nauc_precision_at_3_max
      value: 69.86960822047166
    - type: nauc_precision_at_3_std
      value: -10.230045456623477
    - type: nauc_precision_at_5_diff1
      value: 76.32796311102129
    - type: nauc_precision_at_5_max
      value: 73.88101356144463
    - type: nauc_precision_at_5_std
      value: -6.1510867874405095
    - type: nauc_recall_at_1000_diff1
      value: 73.89525267370503
    - type: nauc_recall_at_1000_max
      value: 83.46793489687492
    - type: nauc_recall_at_1000_std
      value: 43.36957143495507
    - type: nauc_recall_at_100_diff1
      value: 75.33082548811574
    - type: nauc_recall_at_100_max
      value: 85.08356966408451
    - type: nauc_recall_at_100_std
      value: 30.91532174961395
    - type: nauc_recall_at_10_diff1
      value: 75.96120115397329
    - type: nauc_recall_at_10_max
      value: 76.87957072290959
    - type: nauc_recall_at_10_std
      value: 0.4829699864760997
    - type: nauc_recall_at_1_diff1
      value: 83.07021879054804
    - type: nauc_recall_at_1_max
      value: 57.45256621613931
    - type: nauc_recall_at_1_std
      value: -16.94155323782564
    - type: nauc_recall_at_20_diff1
      value: 76.37431221666506
    - type: nauc_recall_at_20_max
      value: 80.17866322470138
    - type: nauc_recall_at_20_std
      value: 12.914278225659201
    - type: nauc_recall_at_3_diff1
      value: 76.55984367763459
    - type: nauc_recall_at_3_max
      value: 69.86960822047196
    - type: nauc_recall_at_3_std
      value: -10.230045456623337
    - type: nauc_recall_at_5_diff1
      value: 76.32796311102106
    - type: nauc_recall_at_5_max
      value: 73.88101356144475
    - type: nauc_recall_at_5_std
      value: -6.151086787440187
    - type: ndcg_at_1
      value: 78.84100000000001
    - type: ndcg_at_10
      value: 87.26899999999999
    - type: ndcg_at_100
      value: 87.977
    - type: ndcg_at_1000
      value: 88.133
    - type: ndcg_at_20
      value: 87.599
    - type: ndcg_at_3
      value: 85.521
    - type: ndcg_at_5
      value: 86.531
    - type: precision_at_1
      value: 78.84100000000001
    - type: precision_at_10
      value: 9.474
    - type: precision_at_100
      value: 0.9809999999999999
    - type: precision_at_1000
      value: 0.099
    - type: precision_at_20
      value: 4.802
    - type: precision_at_3
      value: 30.014000000000003
    - type: precision_at_5
      value: 18.496000000000002
    - type: recall_at_1
      value: 78.84100000000001
    - type: recall_at_10
      value: 94.742
    - type: recall_at_100
      value: 98.056
    - type: recall_at_1000
      value: 99.288
    - type: recall_at_20
      value: 96.04700000000001
    - type: recall_at_3
      value: 90.04100000000001
    - type: recall_at_5
      value: 92.47800000000001
    task:
      type: Retrieval
  - dataset:
      config: php
      name: MTEB COIRCodeSearchNetRetrieval (php)
      revision: 4adc7bc41202b5c13543c9c886a25f340634dab3
      split: test
      type: CoIR-Retrieval/CodeSearchNet
    metrics:
    - type: main_score
      value: 89.384
    - type: map_at_1
      value: 81.997
    - type: map_at_10
      value: 87.221
    - type: map_at_100
      value: 87.375
    - type: map_at_1000
      value: 87.378
    - type: map_at_20
      value: 87.32799999999999
    - type: map_at_3
      value: 86.429
    - type: map_at_5
      value: 86.91600000000001
    - type: mrr_at_1
      value: 81.9965748537177
    - type: mrr_at_10
      value: 87.22106125167394
    - type: mrr_at_100
      value: 87.37512926935148
    - type: mrr_at_1000
      value: 87.37841767615039
    - type: mrr_at_20
      value: 87.32764206073328
    - type: mrr_at_3
      value: 86.42904714333318
    - type: mrr_at_5
      value: 86.9157033442753
    - type: nauc_map_at_1000_diff1
      value: 84.98375890205129
    - type: nauc_map_at_1000_max
      value: 52.169083096188054
    - type: nauc_map_at_1000_std
      value: -35.027524704460916
    - type: nauc_map_at_100_diff1
      value: 84.98366301880897
    - type: nauc_map_at_100_max
      value: 52.177207768384626
    - type: nauc_map_at_100_std
      value: -35.01620836480514
    - type: nauc_map_at_10_diff1
      value: 84.95191430457028
    - type: nauc_map_at_10_max
      value: 52.31158183948366
    - type: nauc_map_at_10_std
      value: -35.13714349074112
    - type: nauc_map_at_1_diff1
      value: 86.2487043950067
    - type: nauc_map_at_1_max
      value: 49.20239525557377
    - type: nauc_map_at_1_std
      value: -33.71661281670688
    - type: nauc_map_at_20_diff1
      value: 84.973933039348
    - type: nauc_map_at_20_max
      value: 52.2266012979943
    - type: nauc_map_at_20_std
      value: -35.04159739087031
    - type: nauc_map_at_3_diff1
      value: 84.69068283876815
    - type: nauc_map_at_3_max
      value: 52.343743449585524
    - type: nauc_map_at_3_std
      value: -35.69366581732263
    - type: nauc_map_at_5_diff1
      value: 84.84059835122619
    - type: nauc_map_at_5_max
      value: 52.29585814594443
    - type: nauc_map_at_5_std
      value: -35.5113867651616
    - type: nauc_mrr_at_1000_diff1
      value: 84.98375890205129
    - type: nauc_mrr_at_1000_max
      value: 52.169083096188054
    - type: nauc_mrr_at_1000_std
      value: -35.027524704460916
    - type: nauc_mrr_at_100_diff1
      value: 84.98366301880897
    - type: nauc_mrr_at_100_max
      value: 52.177207768384626
    - type: nauc_mrr_at_100_std
      value: -35.01620836480514
    - type: nauc_mrr_at_10_diff1
      value: 84.95191430457028
    - type: nauc_mrr_at_10_max
      value: 52.31158183948366
    - type: nauc_mrr_at_10_std
      value: -35.13714349074112
    - type: nauc_mrr_at_1_diff1
      value: 86.2487043950067
    - type: nauc_mrr_at_1_max
      value: 49.20239525557377
    - type: nauc_mrr_at_1_std
      value: -33.71661281670688
    - type: nauc_mrr_at_20_diff1
      value: 84.973933039348
    - type: nauc_mrr_at_20_max
      value: 52.2266012979943
    - type: nauc_mrr_at_20_std
      value: -35.04159739087031
    - type: nauc_mrr_at_3_diff1
      value: 84.69068283876815
    - type: nauc_mrr_at_3_max
      value: 52.343743449585524
    - type: nauc_mrr_at_3_std
      value: -35.69366581732263
    - type: nauc_mrr_at_5_diff1
      value: 84.84059835122619
    - type: nauc_mrr_at_5_max
      value: 52.29585814594443
    - type: nauc_mrr_at_5_std
      value: -35.5113867651616
    - type: nauc_ndcg_at_1000_diff1
      value: 84.8458448603008
    - type: nauc_ndcg_at_1000_max
      value: 52.77266190305249
    - type: nauc_ndcg_at_1000_std
      value: -34.888069489815415
    - type: nauc_ndcg_at_100_diff1
      value: 84.84381394537597
    - type: nauc_ndcg_at_100_max
      value: 53.01563101161627
    - type: nauc_ndcg_at_100_std
      value: -34.509211628586314
    - type: nauc_ndcg_at_10_diff1
      value: 84.66548583706869
    - type: nauc_ndcg_at_10_max
      value: 53.69134601286215
    - type: nauc_ndcg_at_10_std
      value: -35.23415287527687
    - type: nauc_ndcg_at_1_diff1
      value: 86.2487043950067
    - type: nauc_ndcg_at_1_max
      value: 49.20239525557377
    - type: nauc_ndcg_at_1_std
      value: -33.71661281670688
    - type: nauc_ndcg_at_20_diff1
      value: 84.76038572414957
    - type: nauc_ndcg_at_20_max
      value: 53.36813159950785
    - type: nauc_ndcg_at_20_std
      value: -34.821410711460324
    - type: nauc_ndcg_at_3_diff1
      value: 84.1222575178288
    - type: nauc_ndcg_at_3_max
      value: 53.5458597204313
    - type: nauc_ndcg_at_3_std
      value: -36.57805029126134
    - type: nauc_ndcg_at_5_diff1
      value: 84.38081551698716
    - type: nauc_ndcg_at_5_max
      value: 53.5309540463121
    - type: nauc_ndcg_at_5_std
      value: -36.26371526873363
    - type: nauc_precision_at_1000_diff1
      value: 85.52761499735973
    - type: nauc_precision_at_1000_max
      value: 85.57039121692812
    - type: nauc_precision_at_1000_std
      value: 26.076840198712997
    - type: nauc_precision_at_100_diff1
      value: 84.71683047722709
    - type: nauc_precision_at_100_max
      value: 78.6718703783981
    - type: nauc_precision_at_100_std
      value: 7.427629853478685
    - type: nauc_precision_at_10_diff1
      value: 82.66742923426185
    - type: nauc_precision_at_10_max
      value: 65.87095967797794
    - type: nauc_precision_at_10_std
      value: -34.79989719383861
    - type: nauc_precision_at_1_diff1
      value: 86.2487043950067
    - type: nauc_precision_at_1_max
      value: 49.20239525557377
    - type: nauc_precision_at_1_std
      value: -33.71661281670688
    - type: nauc_precision_at_20_diff1
      value: 82.99219098545612
    - type: nauc_precision_at_20_max
      value: 68.30891426964217
    - type: nauc_precision_at_20_std
      value: -28.159133053121295
    - type: nauc_precision_at_3_diff1
      value: 81.61302076796814
    - type: nauc_precision_at_3_max
      value: 58.883669048247576
    - type: nauc_precision_at_3_std
      value: -40.57293703314752
    - type: nauc_precision_at_5_diff1
      value: 81.84587610537042
    - type: nauc_precision_at_5_max
      value: 60.58183720409386
    - type: nauc_precision_at_5_std
      value: -40.54776077088821
    - type: nauc_recall_at_1000_diff1
      value: 85.52761499733693
    - type: nauc_recall_at_1000_max
      value: 85.57039121690828
    - type: nauc_recall_at_1000_std
      value: 26.07684019872863
    - type: nauc_recall_at_100_diff1
      value: 84.7168304772257
    - type: nauc_recall_at_100_max
      value: 78.67187037839709
    - type: nauc_recall_at_100_std
      value: 7.427629853478298
    - type: nauc_recall_at_10_diff1
      value: 82.6674292342617
    - type: nauc_recall_at_10_max
      value: 65.87095967797796
    - type: nauc_recall_at_10_std
      value: -34.7998971938389
    - type: nauc_recall_at_1_diff1
      value: 86.2487043950067
    - type: nauc_recall_at_1_max
      value: 49.20239525557377
    - type: nauc_recall_at_1_std
      value: -33.71661281670688
    - type: nauc_recall_at_20_diff1
      value: 82.9921909854569
    - type: nauc_recall_at_20_max
      value: 68.30891426964256
    - type: nauc_recall_at_20_std
      value: -28.159133053120634
    - type: nauc_recall_at_3_diff1
      value: 81.61302076796801
    - type: nauc_recall_at_3_max
      value: 58.88366904824765
    - type: nauc_recall_at_3_std
      value: -40.57293703314801
    - type: nauc_recall_at_5_diff1
      value: 81.84587610537068
    - type: nauc_recall_at_5_max
      value: 60.5818372040938
    - type: nauc_recall_at_5_std
      value: -40.54776077088785
    - type: ndcg_at_1
      value: 81.997
    - type: ndcg_at_10
      value: 89.384
    - type: ndcg_at_100
      value: 90.091
    - type: ndcg_at_1000
      value: 90.18
    - type: ndcg_at_20
      value: 89.77000000000001
    - type: ndcg_at_3
      value: 87.788
    - type: ndcg_at_5
      value: 88.66000000000001
    - type: precision_at_1
      value: 81.997
    - type: precision_at_10
      value: 9.6
    - type: precision_at_100
      value: 0.992
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.8759999999999994
    - type: precision_at_3
      value: 30.564999999999998
    - type: precision_at_5
      value: 18.759999999999998
    - type: recall_at_1
      value: 81.997
    - type: recall_at_10
      value: 96.004
    - type: recall_at_100
      value: 99.208
    - type: recall_at_1000
      value: 99.914
    - type: recall_at_20
      value: 97.524
    - type: recall_at_3
      value: 91.694
    - type: recall_at_5
      value: 93.79899999999999
    task:
      type: Retrieval
  - dataset:
      config: default
      name: MTEB CodeFeedbackMT (default)
      revision: b0f12fa0c0dd67f59c95a5c33d02aeeb4c398c5f
      split: test
      type: CoIR-Retrieval/codefeedback-mt
    metrics:
    - type: main_score
      value: 93.581
    - type: map_at_1
      value: 89.17699999999999
    - type: map_at_10
      value: 92.293
    - type: map_at_100
      value: 92.39
    - type: map_at_1000
      value: 92.392
    - type: map_at_20
      value: 92.357
    - type: map_at_3
      value: 91.805
    - type: map_at_5
      value: 92.124
    - type: mrr_at_1
      value: 89.1767718611132
    - type: mrr_at_10
      value: 92.29094710867727
    - type: mrr_at_100
      value: 92.38822901927145
    - type: mrr_at_1000
      value: 92.39035562398948
    - type: mrr_at_20
      value: 92.35530678666272
    - type: mrr_at_3
      value: 91.80412241721295
    - type: mrr_at_5
      value: 92.122718485602
    - type: nauc_map_at_1000_diff1
      value: 91.32273096224377
    - type: nauc_map_at_1000_max
      value: 67.91710992417916
    - type: nauc_map_at_1000_std
      value: -74.42383054373403
    - type: nauc_map_at_100_diff1
      value: 91.32246480919676
    - type: nauc_map_at_100_max
      value: 67.91988217232155
    - type: nauc_map_at_100_std
      value: -74.42982893903614
    - type: nauc_map_at_10_diff1
      value: 91.33091844723225
    - type: nauc_map_at_10_max
      value: 67.99200052857573
    - type: nauc_map_at_10_std
      value: -74.86264731079909
    - type: nauc_map_at_1_diff1
      value: 91.19451424067374
    - type: nauc_map_at_1_max
      value: 66.25392135499182
    - type: nauc_map_at_1_std
      value: -66.71862444546531
    - type: nauc_map_at_20_diff1
      value: 91.338096532563
    - type: nauc_map_at_20_max
      value: 67.9501381727388
    - type: nauc_map_at_20_std
      value: -74.6541487409306
    - type: nauc_map_at_3_diff1
      value: 91.33378646831018
    - type: nauc_map_at_3_max
      value: 68.07178423980443
    - type: nauc_map_at_3_std
      value: -74.8448215489277
    - type: nauc_map_at_5_diff1
      value: 91.31919404747947
    - type: nauc_map_at_5_max
      value: 67.9846196631338
    - type: nauc_map_at_5_std
      value: -75.1772163451522
    - type: nauc_mrr_at_1000_diff1
      value: 91.3199902377502
    - type: nauc_mrr_at_1000_max
      value: 67.9218672510084
    - type: nauc_mrr_at_1000_std
      value: -74.43167854141912
    - type: nauc_mrr_at_100_diff1
      value: 91.31972491331909
    - type: nauc_mrr_at_100_max
      value: 67.92463756203038
    - type: nauc_mrr_at_100_std
      value: -74.43767343342054
    - type: nauc_mrr_at_10_diff1
      value: 91.32821156688269
    - type: nauc_mrr_at_10_max
      value: 67.9966801151313
    - type: nauc_mrr_at_10_std
      value: -74.87029985666574
    - type: nauc_mrr_at_1_diff1
      value: 91.19451424067374
    - type: nauc_mrr_at_1_max
      value: 66.25392135499182
    - type: nauc_mrr_at_1_std
      value: -66.71862444546531
    - type: nauc_mrr_at_20_diff1
      value: 91.33536515912955
    - type: nauc_mrr_at_20_max
      value: 67.95486643404077
    - type: nauc_mrr_at_20_std
      value: -74.66191099031441
    - type: nauc_mrr_at_3_diff1
      value: 91.330840031252
    - type: nauc_mrr_at_3_max
      value: 68.07469130953064
    - type: nauc_mrr_at_3_std
      value: -74.86010698346193
    - type: nauc_mrr_at_5_diff1
      value: 91.31654781777661
    - type: nauc_mrr_at_5_max
      value: 67.98920016571047
    - type: nauc_mrr_at_5_std
      value: -75.18463872642327
    - type: nauc_ndcg_at_1000_diff1
      value: 91.34667276431432
    - type: nauc_ndcg_at_1000_max
      value: 68.21491424591551
    - type: nauc_ndcg_at_1000_std
      value: -75.66992744128785
    - type: nauc_ndcg_at_100_diff1
      value: 91.32392901146443
    - type: nauc_ndcg_at_100_max
      value: 68.29729595248577
    - type: nauc_ndcg_at_100_std
      value: -75.71771214390895
    - type: nauc_ndcg_at_10_diff1
      value: 91.42077595372994
    - type: nauc_ndcg_at_10_max
      value: 68.68694743767313
    - type: nauc_ndcg_at_10_std
      value: -78.47824539632552
    - type: nauc_ndcg_at_1_diff1
      value: 91.19451424067374
    - type: nauc_ndcg_at_1_max
      value: 66.25392135499182
    - type: nauc_ndcg_at_1_std
      value: -66.71862444546531
    - type: nauc_ndcg_at_20_diff1
      value: 91.45155994342801
    - type: nauc_ndcg_at_20_max
      value: 68.53325586053745
    - type: nauc_ndcg_at_20_std
      value: -77.61552755050272
    - type: nauc_ndcg_at_3_diff1
      value: 91.40048288131362
    - type: nauc_ndcg_at_3_max
      value: 68.77879241382503
    - type: nauc_ndcg_at_3_std
      value: -78.145772024284
    - type: nauc_ndcg_at_5_diff1
      value: 91.38284059773454
    - type: nauc_ndcg_at_5_max
      value: 68.65210990623613
    - type: nauc_ndcg_at_5_std
      value: -79.12029324998576
    - type: nauc_precision_at_1000_diff1
      value: 93.46388819144805
    - type: nauc_precision_at_1000_max
      value: 100.0
    - type: nauc_precision_at_1000_std
      value: 63.81917689216168
    - type: nauc_precision_at_100_diff1
      value: 88.14968521318447
    - type: nauc_precision_at_100_max
      value: 79.85443666227313
    - type: nauc_precision_at_100_std
      value: -71.60524535547103
    - type: nauc_precision_at_10_diff1
      value: 92.26837092071085
    - type: nauc_precision_at_10_max
      value: 74.6573995994554
    - type: nauc_precision_at_10_std
      value: -110.01042456501374
    - type: nauc_precision_at_1_diff1
      value: 91.19451424067374
    - type: nauc_precision_at_1_max
      value: 66.25392135499182
    - type: nauc_precision_at_1_std
      value: -66.71862444546531
    - type: nauc_precision_at_20_diff1
      value: 93.20186301743918
    - type: nauc_precision_at_20_max
      value: 75.80862507034404
    - type: nauc_precision_at_20_std
      value: -114.47170564758076
    - type: nauc_precision_at_3_diff1
      value: 91.7095571991953
    - type: nauc_precision_at_3_max
      value: 71.9834034178247
    - type: nauc_precision_at_3_std
      value: -93.18361438847609
    - type: nauc_precision_at_5_diff1
      value: 91.7773542794046
    - type: nauc_precision_at_5_max
      value: 72.60584615957593
    - type: nauc_precision_at_5_std
      value: -103.21989758142311
    - type: nauc_recall_at_1000_diff1
      value: 93.46388819147347
    - type: nauc_recall_at_1000_max
      value: 100.0
    - type: nauc_recall_at_1000_std
      value: 63.819176892126386
    - type: nauc_recall_at_100_diff1
      value: 88.14968521317887
    - type: nauc_recall_at_100_max
      value: 79.85443666226894
    - type: nauc_recall_at_100_std
      value: -71.60524535547005
    - type: nauc_recall_at_10_diff1
      value: 92.2683709207107
    - type: nauc_recall_at_10_max
      value: 74.65739959945586
    - type: nauc_recall_at_10_std
      value: -110.01042456501415
    - type: nauc_recall_at_1_diff1
      value: 91.19451424067374
    - type: nauc_recall_at_1_max
      value: 66.25392135499182
    - type: nauc_recall_at_1_std
      value: -66.71862444546531
    - type: nauc_recall_at_20_diff1
      value: 93.20186301743836
    - type: nauc_recall_at_20_max
      value: 75.8086250703445
    - type: nauc_recall_at_20_std
      value: -114.47170564758093
    - type: nauc_recall_at_3_diff1
      value: 91.70955719919579
    - type: nauc_recall_at_3_max
      value: 71.9834034178246
    - type: nauc_recall_at_3_std
      value: -93.1836143884767
    - type: nauc_recall_at_5_diff1
      value: 91.77735427940513
    - type: nauc_recall_at_5_max
      value: 72.60584615957615
    - type: nauc_recall_at_5_std
      value: -103.21989758142347
    - type: ndcg_at_1
      value: 89.17699999999999
    - type: ndcg_at_10
      value: 93.581
    - type: ndcg_at_100
      value: 94.041
    - type: ndcg_at_1000
      value: 94.087
    - type: ndcg_at_20
      value: 93.813
    - type: ndcg_at_3
      value: 92.608
    - type: ndcg_at_5
      value: 93.183
    - type: precision_at_1
      value: 89.17699999999999
    - type: precision_at_10
      value: 9.751999999999999
    - type: precision_at_100
      value: 0.996
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.922
    - type: precision_at_3
      value: 31.639
    - type: precision_at_5
      value: 19.262
    - type: recall_at_1
      value: 89.17699999999999
    - type: recall_at_10
      value: 97.52199999999999
    - type: recall_at_100
      value: 99.638
    - type: recall_at_1000
      value: 99.985
    - type: recall_at_20
      value: 98.433
    - type: recall_at_3
      value: 94.916
    - type: recall_at_5
      value: 96.309
    task:
      type: Retrieval
  - dataset:
      config: default
      name: MTEB CodeFeedbackST (default)
      revision: d213819e87aab9010628da8b73ab4eb337c89340
      split: test
      type: CoIR-Retrieval/codefeedback-st
    metrics:
    - type: main_score
      value: 90.672
    - type: map_at_1
      value: 80.17
    - type: map_at_10
      value: 87.945
    - type: map_at_100
      value: 88.004
    - type: map_at_1000
      value: 88.004
    - type: map_at_20
      value: 87.99
    - type: map_at_3
      value: 87.248
    - type: map_at_5
      value: 87.79599999999999
    - type: mrr_at_1
      value: 80.08369002747078
    - type: mrr_at_10
      value: 87.8921544021677
    - type: mrr_at_100
      value: 87.95039935983601
    - type: mrr_at_1000
      value: 87.95113333851275
    - type: mrr_at_20
      value: 87.93646217104668
    - type: mrr_at_3
      value: 87.1787090866489
    - type: mrr_at_5
      value: 87.73579079622561
    - type: nauc_map_at_1000_diff1
      value: 84.59599547053739
    - type: nauc_map_at_1000_max
      value: 11.563061625907412
    - type: nauc_map_at_1000_std
      value: -60.044443468702234
    - type: nauc_map_at_100_diff1
      value: 84.59537761332842
    - type: nauc_map_at_100_max
      value: 11.565642697551821
    - type: nauc_map_at_100_std
      value: -60.04854322246466
    - type: nauc_map_at_10_diff1
      value: 84.54000249494548
    - type: nauc_map_at_10_max
      value: 11.55440805584509
    - type: nauc_map_at_10_std
      value: -60.437831701830824
    - type: nauc_map_at_1_diff1
      value: 86.11748953192901
    - type: nauc_map_at_1_max
      value: 11.5654030601593
    - type: nauc_map_at_1_std
      value: -53.511088508152106
    - type: nauc_map_at_20_diff1
      value: 84.58313379439
    - type: nauc_map_at_20_max
      value: 11.570044402077079
    - type: nauc_map_at_20_std
      value: -60.15198207708651
    - type: nauc_map_at_3_diff1
      value: 84.32880719653019
    - type: nauc_map_at_3_max
      value: 11.306870549359703
    - type: nauc_map_at_3_std
      value: -61.66972599149546
    - type: nauc_map_at_5_diff1
      value: 84.44481140471144
    - type: nauc_map_at_5_max
      value: 11.431107322656027
    - type: nauc_map_at_5_std
      value: -61.07717915800889
    - type: nauc_mrr_at_1000_diff1
      value: 84.68763078545194
    - type: nauc_mrr_at_1000_max
      value: 11.654873092581681
    - type: nauc_mrr_at_1000_std
      value: -59.336986725737205
    - type: nauc_mrr_at_100_diff1
      value: 84.68700936878848
    - type: nauc_mrr_at_100_max
      value: 11.657439639783314
    - type: nauc_mrr_at_100_std
      value: -59.34111569519106
    - type: nauc_mrr_at_10_diff1
      value: 84.6310183439079
    - type: nauc_mrr_at_10_max
      value: 11.644216926632295
    - type: nauc_mrr_at_10_std
      value: -59.72967384330525
    - type: nauc_mrr_at_1_diff1
      value: 86.20926531200153
    - type: nauc_mrr_at_1_max
      value: 11.798672092672408
    - type: nauc_mrr_at_1_std
      value: -52.65061182155048
    - type: nauc_mrr_at_20_diff1
      value: 84.67473718323151
    - type: nauc_mrr_at_20_max
      value: 11.66210308966798
    - type: nauc_mrr_at_20_std
      value: -59.44638680106128
    - type: nauc_mrr_at_3_diff1
      value: 84.43542937601633
    - type: nauc_mrr_at_3_max
      value: 11.386252739390274
    - type: nauc_mrr_at_3_std
      value: -60.847260123302824
    - type: nauc_mrr_at_5_diff1
      value: 84.54163679900752
    - type: nauc_mrr_at_5_max
      value: 11.507249429826652
    - type: nauc_mrr_at_5_std
      value: -60.37227397638133
    - type: nauc_ndcg_at_1000_diff1
      value: 84.38515659117648
    - type: nauc_ndcg_at_1000_max
      value: 11.696388104993815
    - type: nauc_ndcg_at_1000_std
      value: -60.99717188632636
    - type: nauc_ndcg_at_100_diff1
      value: 84.36252666208128
    - type: nauc_ndcg_at_100_max
      value: 11.779193272629582
    - type: nauc_ndcg_at_100_std
      value: -61.07767732450027
    - type: nauc_ndcg_at_10_diff1
      value: 84.04954066703365
    - type: nauc_ndcg_at_10_max
      value: 11.779974489837082
    - type: nauc_ndcg_at_10_std
      value: -63.216690095927774
    - type: nauc_ndcg_at_1_diff1
      value: 86.11748953192901
    - type: nauc_ndcg_at_1_max
      value: 11.5654030601593
    - type: nauc_ndcg_at_1_std
      value: -53.511088508152106
    - type: nauc_ndcg_at_20_diff1
      value: 84.24823772848971
    - type: nauc_ndcg_at_20_max
      value: 11.863360518928207
    - type: nauc_ndcg_at_20_std
      value: -61.93361896101012
    - type: nauc_ndcg_at_3_diff1
      value: 83.59193584331861
    - type: nauc_ndcg_at_3_max
      value: 11.101961347273388
    - type: nauc_ndcg_at_3_std
      value: -65.94436298501897
    - type: nauc_ndcg_at_5_diff1
      value: 83.78223501020723
    - type: nauc_ndcg_at_5_max
      value: 11.381166299845784
    - type: nauc_ndcg_at_5_std
      value: -65.04889907197641
    - type: nauc_precision_at_1000_diff1
      value: 76.06066445979025
    - type: nauc_precision_at_1000_max
      value: 88.80070662604956
    - type: nauc_precision_at_1000_std
      value: 52.334059183043905
    - type: nauc_precision_at_100_diff1
      value: 72.04652343451335
    - type: nauc_precision_at_100_max
      value: 63.89605474602823
    - type: nauc_precision_at_100_std
      value: -69.43370560244632
    - type: nauc_precision_at_10_diff1
      value: 72.89952199107627
    - type: nauc_precision_at_10_max
      value: 19.311114535110576
    - type: nauc_precision_at_10_std
      value: -128.84033294430404
    - type: nauc_precision_at_1_diff1
      value: 86.11748953192901
    - type: nauc_precision_at_1_max
      value: 11.5654030601593
    - type: nauc_precision_at_1_std
      value: -53.511088508152106
    - type: nauc_precision_at_20_diff1
      value: 72.67081355507392
    - type: nauc_precision_at_20_max
      value: 31.810009027751327
    - type: nauc_precision_at_20_std
      value: -125.0241647763245
    - type: nauc_precision_at_3_diff1
      value: 78.25481403127574
    - type: nauc_precision_at_3_max
      value: 9.522751941864358
    - type: nauc_precision_at_3_std
      value: -97.78790351049261
    - type: nauc_precision_at_5_diff1
      value: 74.76998378870415
    - type: nauc_precision_at_5_max
      value: 10.813166077171799
    - type: nauc_precision_at_5_std
      value: -121.15471825739507
    - type: nauc_recall_at_1000_diff1
      value: 76.06066445979108
    - type: nauc_recall_at_1000_max
      value: 88.80070662604261
    - type: nauc_recall_at_1000_std
      value: 52.334059183053746
    - type: nauc_recall_at_100_diff1
      value: 72.04652343451711
    - type: nauc_recall_at_100_max
      value: 63.896054746020795
    - type: nauc_recall_at_100_std
      value: -69.43370560243693
    - type: nauc_recall_at_10_diff1
      value: 72.89952199107772
    - type: nauc_recall_at_10_max
      value: 19.311114535110473
    - type: nauc_recall_at_10_std
      value: -128.8403329443047
    - type: nauc_recall_at_1_diff1
      value: 86.11748953192901
    - type: nauc_recall_at_1_max
      value: 11.5654030601593
    - type: nauc_recall_at_1_std
      value: -53.511088508152106
    - type: nauc_recall_at_20_diff1
      value: 72.6708135550744
    - type: nauc_recall_at_20_max
      value: 31.81000902775227
    - type: nauc_recall_at_20_std
      value: -125.02416477631772
    - type: nauc_recall_at_3_diff1
      value: 78.25481403127577
    - type: nauc_recall_at_3_max
      value: 9.522751941864486
    - type: nauc_recall_at_3_std
      value: -97.78790351049261
    - type: nauc_recall_at_5_diff1
      value: 74.76998378870401
    - type: nauc_recall_at_5_max
      value: 10.813166077172303
    - type: nauc_recall_at_5_std
      value: -121.15471825739255
    - type: ndcg_at_1
      value: 80.17
    - type: ndcg_at_10
      value: 90.672
    - type: ndcg_at_100
      value: 90.923
    - type: ndcg_at_1000
      value: 90.93900000000001
    - type: ndcg_at_20
      value: 90.833
    - type: ndcg_at_3
      value: 89.347
    - type: ndcg_at_5
      value: 90.321
    - type: precision_at_1
      value: 80.17
    - type: precision_at_10
      value: 9.875
    - type: precision_at_100
      value: 0.9990000000000001
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.968999999999999
    - type: precision_at_3
      value: 31.785999999999998
    - type: precision_at_5
      value: 19.538
    - type: recall_at_1
      value: 80.17
    - type: recall_at_10
      value: 98.75399999999999
    - type: recall_at_100
      value: 99.85300000000001
    - type: recall_at_1000
      value: 99.971
    - type: recall_at_20
      value: 99.384
    - type: recall_at_3
      value: 95.35900000000001
    - type: recall_at_5
      value: 97.69099999999999
    task:
      type: Retrieval
  - dataset:
      config: python
      name: MTEB CodeSearchNetCCRetrieval (python)
      revision: 6e1effa2c03723c5fde48ee912b5ee08d4f211e8
      split: test
      type: CoIR-Retrieval/CodeSearchNet-ccr
    metrics:
    - type: main_score
      value: 94.433
    - type: map_at_1
      value: 89.19399999999999
    - type: map_at_10
      value: 93.028
    - type: map_at_100
      value: 93.082
    - type: map_at_1000
      value: 93.083
    - type: map_at_20
      value: 93.063
    - type: map_at_3
      value: 92.635
    - type: map_at_5
      value: 92.914
    - type: mrr_at_1
      value: 89.19426196541092
    - type: mrr_at_10
      value: 93.02709840248427
    - type: mrr_at_100
      value: 93.08116003424139
    - type: mrr_at_1000
      value: 93.08217593910118
    - type: mrr_at_20
      value: 93.06190896089124
    - type: mrr_at_3
      value: 92.63529516914724
    - type: mrr_at_5
      value: 92.91147160030434
    - type: nauc_map_at_1000_diff1
      value: 88.83311452663212
    - type: nauc_map_at_1000_max
      value: 60.95017820522245
    - type: nauc_map_at_1000_std
      value: -27.89769485046986
    - type: nauc_map_at_100_diff1
      value: 88.83290610426658
    - type: nauc_map_at_100_max
      value: 60.95500551380402
    - type: nauc_map_at_100_std
      value: -27.89016844487898
    - type: nauc_map_at_10_diff1
      value: 88.85572581806649
    - type: nauc_map_at_10_max
      value: 61.15711675036097
    - type: nauc_map_at_10_std
      value: -27.863737251156817
    - type: nauc_map_at_1_diff1
      value: 88.75783103896558
    - type: nauc_map_at_1_max
      value: 56.05082536501996
    - type: nauc_map_at_1_std
      value: -26.762271272974058
    - type: nauc_map_at_20_diff1
      value: 88.83684101691745
    - type: nauc_map_at_20_max
      value: 61.005770466214415
    - type: nauc_map_at_20_std
      value: -27.93102109901855
    - type: nauc_map_at_3_diff1
      value: 88.8117321224026
    - type: nauc_map_at_3_max
      value: 61.93756889954432
    - type: nauc_map_at_3_std
      value: -28.688755234894575
    - type: nauc_map_at_5_diff1
      value: 88.80829639130586
    - type: nauc_map_at_5_max
      value: 61.453270288714315
    - type: nauc_map_at_5_std
      value: -28.349980140724064
    - type: nauc_mrr_at_1000_diff1
      value: 88.84020090540668
    - type: nauc_mrr_at_1000_max
      value: 60.889945118911925
    - type: nauc_mrr_at_1000_std
      value: -27.98205495040352
    - type: nauc_mrr_at_100_diff1
      value: 88.8399914575814
    - type: nauc_mrr_at_100_max
      value: 60.894780806340286
    - type: nauc_mrr_at_100_std
      value: -27.974516894245998
    - type: nauc_mrr_at_10_diff1
      value: 88.86278756786537
    - type: nauc_mrr_at_10_max
      value: 61.097374379118406
    - type: nauc_mrr_at_10_std
      value: -27.94782127554896
    - type: nauc_mrr_at_1_diff1
      value: 88.75783103896558
    - type: nauc_mrr_at_1_max
      value: 56.00537716973826
    - type: nauc_mrr_at_1_std
      value: -26.855708242461617
    - type: nauc_mrr_at_20_diff1
      value: 88.84394060587955
    - type: nauc_mrr_at_20_max
      value: 60.94574319151849
    - type: nauc_mrr_at_20_std
      value: -28.015522604336574
    - type: nauc_mrr_at_3_diff1
      value: 88.8197212493924
    - type: nauc_mrr_at_3_max
      value: 61.864940406758166
    - type: nauc_mrr_at_3_std
      value: -28.785412568295914
    - type: nauc_mrr_at_5_diff1
      value: 88.82086414470385
    - type: nauc_mrr_at_5_max
      value: 61.39568301316898
    - type: nauc_mrr_at_5_std
      value: -28.470440741112203
    - type: nauc_ndcg_at_1000_diff1
      value: 88.87772673122608
    - type: nauc_ndcg_at_1000_max
      value: 61.72947423485932
    - type: nauc_ndcg_at_1000_std
      value: -27.73716470116156
    - type: nauc_ndcg_at_100_diff1
      value: 88.88494233648349
    - type: nauc_ndcg_at_100_max
      value: 61.89646036981659
    - type: nauc_ndcg_at_100_std
      value: -27.485345609558276
    - type: nauc_ndcg_at_10_diff1
      value: 89.0023293800841
    - type: nauc_ndcg_at_10_max
      value: 62.94119991860201
    - type: nauc_ndcg_at_10_std
      value: -27.460194148873917
    - type: nauc_ndcg_at_1_diff1
      value: 88.75783103896558
    - type: nauc_ndcg_at_1_max
      value: 56.05082536501996
    - type: nauc_ndcg_at_1_std
      value: -26.762271272974058
    - type: nauc_ndcg_at_20_diff1
      value: 88.91820321119603
    - type: nauc_ndcg_at_20_max
      value: 62.296584649286466
    - type: nauc_ndcg_at_20_std
      value: -27.766244930438113
    - type: nauc_ndcg_at_3_diff1
      value: 88.8645684782722
    - type: nauc_ndcg_at_3_max
      value: 64.56115657789144
    - type: nauc_ndcg_at_3_std
      value: -29.624561516864517
    - type: nauc_ndcg_at_5_diff1
      value: 88.86373245096877
    - type: nauc_ndcg_at_5_max
      value: 63.72960904786197
    - type: nauc_ndcg_at_5_std
      value: -28.9714090724707
    - type: nauc_precision_at_1000_diff1
      value: 93.46395492502747
    - type: nauc_precision_at_1000_max
      value: 100.0
    - type: nauc_precision_at_1000_std
      value: 79.16615094256558
    - type: nauc_precision_at_100_diff1
      value: 91.18620004808257
    - type: nauc_precision_at_100_max
      value: 93.22441208688358
    - type: nauc_precision_at_100_std
      value: 24.541027605722128
    - type: nauc_precision_at_10_diff1
      value: 91.39898569182662
    - type: nauc_precision_at_10_max
      value: 86.36227958399685
    - type: nauc_precision_at_10_std
      value: -18.10242240819358
    - type: nauc_precision_at_1_diff1
      value: 88.75783103896558
    - type: nauc_precision_at_1_max
      value: 56.05082536501996
    - type: nauc_precision_at_1_std
      value: -26.762271272974058
    - type: nauc_precision_at_20_diff1
      value: 90.67031369822483
    - type: nauc_precision_at_20_max
      value: 83.74877222150064
    - type: nauc_precision_at_20_std
      value: -20.408972788884217
    - type: nauc_precision_at_3_diff1
      value: 89.20609963322217
    - type: nauc_precision_at_3_max
      value: 80.51313561745297
    - type: nauc_precision_at_3_std
      value: -35.36879832053998
    - type: nauc_precision_at_5_diff1
      value: 89.40146615001582
    - type: nauc_precision_at_5_max
      value: 83.94859838264775
    - type: nauc_precision_at_5_std
      value: -34.3716686751886
    - type: nauc_recall_at_1000_diff1
      value: 93.46395492500885
    - type: nauc_recall_at_1000_max
      value: 100.0
    - type: nauc_recall_at_1000_std
      value: 79.1661509425585
    - type: nauc_recall_at_100_diff1
      value: 91.18620004808852
    - type: nauc_recall_at_100_max
      value: 93.22441208689622
    - type: nauc_recall_at_100_std
      value: 24.54102760573055
    - type: nauc_recall_at_10_diff1
      value: 91.39898569182566
    - type: nauc_recall_at_10_max
      value: 86.36227958399564
    - type: nauc_recall_at_10_std
      value: -18.102422408192854
    - type: nauc_recall_at_1_diff1
      value: 88.75783103896558
    - type: nauc_recall_at_1_max
      value: 56.05082536501996
    - type: nauc_recall_at_1_std
      value: -26.762271272974058
    - type: nauc_recall_at_20_diff1
      value: 90.67031369822419
    - type: nauc_recall_at_20_max
      value: 83.7487722215012
    - type: nauc_recall_at_20_std
      value: -20.408972788880643
    - type: nauc_recall_at_3_diff1
      value: 89.20609963322205
    - type: nauc_recall_at_3_max
      value: 80.51313561745232
    - type: nauc_recall_at_3_std
      value: -35.36879832053958
    - type: nauc_recall_at_5_diff1
      value: 89.40146615001574
    - type: nauc_recall_at_5_max
      value: 83.94859838264772
    - type: nauc_recall_at_5_std
      value: -34.371668675188346
    - type: ndcg_at_1
      value: 89.19399999999999
    - type: ndcg_at_10
      value: 94.433
    - type: ndcg_at_100
      value: 94.679
    - type: ndcg_at_1000
      value: 94.707
    - type: ndcg_at_20
      value: 94.556
    - type: ndcg_at_3
      value: 93.661
    - type: ndcg_at_5
      value: 94.161
    - type: precision_at_1
      value: 89.19399999999999
    - type: precision_at_10
      value: 9.864
    - type: precision_at_100
      value: 0.997
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.956
    - type: precision_at_3
      value: 32.2
    - type: precision_at_5
      value: 19.561999999999998
    - type: recall_at_1
      value: 89.19399999999999
    - type: recall_at_10
      value: 98.639
    - type: recall_at_100
      value: 99.74499999999999
    - type: recall_at_1000
      value: 99.973
    - type: recall_at_20
      value: 99.122
    - type: recall_at_3
      value: 96.601
    - type: recall_at_5
      value: 97.80799999999999
    task:
      type: Retrieval
  - dataset:
      config: javascript
      name: MTEB CodeSearchNetCCRetrieval (javascript)
      revision: 6e1effa2c03723c5fde48ee912b5ee08d4f211e8
      split: test
      type: CoIR-Retrieval/CodeSearchNet-ccr
    metrics:
    - type: main_score
      value: 91.532
    - type: map_at_1
      value: 84.04700000000001
    - type: map_at_10
      value: 89.50399999999999
    - type: map_at_100
      value: 89.581
    - type: map_at_1000
      value: 89.584
    - type: map_at_20
      value: 89.548
    - type: map_at_3
      value: 88.914
    - type: map_at_5
      value: 89.271
    - type: mrr_at_1
      value: 83.98663020358553
    - type: mrr_at_10
      value: 89.46699271220692
    - type: mrr_at_100
      value: 89.54429619092761
    - type: mrr_at_1000
      value: 89.54715966900696
    - type: mrr_at_20
      value: 89.51131350599995
    - type: mrr_at_3
      value: 88.8787602552416
    - type: mrr_at_5
      value: 89.23427529626254
    - type: nauc_map_at_1000_diff1
      value: 81.33895244348352
    - type: nauc_map_at_1000_max
      value: 51.005827566402075
    - type: nauc_map_at_1000_std
      value: -22.252578589937457
    - type: nauc_map_at_100_diff1
      value: 81.33433499029653
    - type: nauc_map_at_100_max
      value: 51.01323653352282
    - type: nauc_map_at_100_std
      value: -22.24639503900029
    - type: nauc_map_at_10_diff1
      value: 81.30647802788602
    - type: nauc_map_at_10_max
      value: 51.211145238065484
    - type: nauc_map_at_10_std
      value: -22.30319656889033
    - type: nauc_map_at_1_diff1
      value: 82.57372620311462
    - type: nauc_map_at_1_max
      value: 44.54905481211588
    - type: nauc_map_at_1_std
      value: -21.962071475865756
    - type: nauc_map_at_20_diff1
      value: 81.31928022528241
    - type: nauc_map_at_20_max
      value: 51.13238604171399
    - type: nauc_map_at_20_std
      value: -22.20703220818948
    - type: nauc_map_at_3_diff1
      value: 81.10849311491785
    - type: nauc_map_at_3_max
      value: 51.8208019735269
    - type: nauc_map_at_3_std
      value: -22.96789103735518
    - type: nauc_map_at_5_diff1
      value: 81.1494438340007
    - type: nauc_map_at_5_max
      value: 51.54484151827443
    - type: nauc_map_at_5_std
      value: -22.39835418757183
    - type: nauc_mrr_at_1000_diff1
      value: 81.42887177105008
    - type: nauc_mrr_at_1000_max
      value: 50.78456409105162
    - type: nauc_mrr_at_1000_std
      value: -22.810803121918408
    - type: nauc_mrr_at_100_diff1
      value: 81.42424440922578
    - type: nauc_mrr_at_100_max
      value: 50.792010710475864
    - type: nauc_mrr_at_100_std
      value: -22.804483531306143
    - type: nauc_mrr_at_10_diff1
      value: 81.39579265182797
    - type: nauc_mrr_at_10_max
      value: 50.99096194207801
    - type: nauc_mrr_at_10_std
      value: -22.856845472961556
    - type: nauc_mrr_at_1_diff1
      value: 82.68635369279959
    - type: nauc_mrr_at_1_max
      value: 44.2393421717323
    - type: nauc_mrr_at_1_std
      value: -22.775172610293808
    - type: nauc_mrr_at_20_diff1
      value: 81.40894511054444
    - type: nauc_mrr_at_20_max
      value: 50.91149353347425
    - type: nauc_mrr_at_20_std
      value: -22.763426041817088
    - type: nauc_mrr_at_3_diff1
      value: 81.1947777094554
    - type: nauc_mrr_at_3_max
      value: 51.60584449653166
    - type: nauc_mrr_at_3_std
      value: -23.48440779958821
    - type: nauc_mrr_at_5_diff1
      value: 81.23725416306374
    - type: nauc_mrr_at_5_max
      value: 51.32853667650834
    - type: nauc_mrr_at_5_std
      value: -22.939278713896133
    - type: nauc_ndcg_at_1000_diff1
      value: 81.12746150854298
    - type: nauc_ndcg_at_1000_max
      value: 52.1786963598477
    - type: nauc_ndcg_at_1000_std
      value: -21.66735535060729
    - type: nauc_ndcg_at_100_diff1
      value: 81.00850024967238
    - type: nauc_ndcg_at_100_max
      value: 52.376900809779855
    - type: nauc_ndcg_at_100_std
      value: -21.437187821641885
    - type: nauc_ndcg_at_10_diff1
      value: 80.87079398487964
    - type: nauc_ndcg_at_10_max
      value: 53.719012858465916
    - type: nauc_ndcg_at_10_std
      value: -21.57272074203413
    - type: nauc_ndcg_at_1_diff1
      value: 82.57372620311462
    - type: nauc_ndcg_at_1_max
      value: 44.54905481211588
    - type: nauc_ndcg_at_1_std
      value: -21.962071475865756
    - type: nauc_ndcg_at_20_diff1
      value: 80.89584525085091
    - type: nauc_ndcg_at_20_max
      value: 53.401415874657296
    - type: nauc_ndcg_at_20_std
      value: -21.129352577941283
    - type: nauc_ndcg_at_3_diff1
      value: 80.46103065917166
    - type: nauc_ndcg_at_3_max
      value: 54.91263156860439
    - type: nauc_ndcg_at_3_std
      value: -23.0656484747823
    - type: nauc_ndcg_at_5_diff1
      value: 80.50385469743831
    - type: nauc_ndcg_at_5_max
      value: 54.53609881176663
    - type: nauc_ndcg_at_5_std
      value: -21.826145336827878
    - type: nauc_precision_at_1000_diff1
      value: 81.4825203906492
    - type: nauc_precision_at_1000_max
      value: 100.0
    - type: nauc_precision_at_1000_std
      value: 100.0
    - type: nauc_precision_at_100_diff1
      value: 69.4979677737838
    - type: nauc_precision_at_100_max
      value: 79.81453242811645
    - type: nauc_precision_at_100_std
      value: 22.735882735419853
    - type: nauc_precision_at_10_diff1
      value: 76.02578489916326
    - type: nauc_precision_at_10_max
      value: 82.20220582823438
    - type: nauc_precision_at_10_std
      value: -11.133888291415301
    - type: nauc_precision_at_1_diff1
      value: 82.57372620311462
    - type: nauc_precision_at_1_max
      value: 44.54905481211588
    - type: nauc_precision_at_1_std
      value: -21.962071475865756
    - type: nauc_precision_at_20_diff1
      value: 74.57535506743962
    - type: nauc_precision_at_20_max
      value: 86.22474553094298
    - type: nauc_precision_at_20_std
      value: 0.37124480616714156
    - type: nauc_precision_at_3_diff1
      value: 76.90725827318549
    - type: nauc_precision_at_3_max
      value: 71.80786851596808
    - type: nauc_precision_at_3_std
      value: -23.38779609088237
    - type: nauc_precision_at_5_diff1
      value: 75.76703985132757
    - type: nauc_precision_at_5_max
      value: 76.30757619721807
    - type: nauc_precision_at_5_std
      value: -16.772096432698
    - type: nauc_recall_at_1000_diff1
      value: 81.48252039064056
    - type: nauc_recall_at_1000_max
      value: 100.0
    - type: nauc_recall_at_1000_std
      value: 100.0
    - type: nauc_recall_at_100_diff1
      value: 69.49796777378361
    - type: nauc_recall_at_100_max
      value: 79.81453242811749
    - type: nauc_recall_at_100_std
      value: 22.73588273542357
    - type: nauc_recall_at_10_diff1
      value: 76.0257848991628
    - type: nauc_recall_at_10_max
      value: 82.20220582823433
    - type: nauc_recall_at_10_std
      value: -11.13388829141429
    - type: nauc_recall_at_1_diff1
      value: 82.57372620311462
    - type: nauc_recall_at_1_max
      value: 44.54905481211588
    - type: nauc_recall_at_1_std
      value: -21.962071475865756
    - type: nauc_recall_at_20_diff1
      value: 74.57535506744009
    - type: nauc_recall_at_20_max
      value: 86.22474553094256
    - type: nauc_recall_at_20_std
      value: 0.3712448061678841
    - type: nauc_recall_at_3_diff1
      value: 76.90725827318568
    - type: nauc_recall_at_3_max
      value: 71.80786851596824
    - type: nauc_recall_at_3_std
      value: -23.38779609088207
    - type: nauc_recall_at_5_diff1
      value: 75.76703985132794
    - type: nauc_recall_at_5_max
      value: 76.30757619721837
    - type: nauc_recall_at_5_std
      value: -16.772096432697477
    - type: ndcg_at_1
      value: 84.04700000000001
    - type: ndcg_at_10
      value: 91.532
    - type: ndcg_at_100
      value: 91.914
    - type: ndcg_at_1000
      value: 91.979
    - type: ndcg_at_20
      value: 91.688
    - type: ndcg_at_3
      value: 90.35300000000001
    - type: ndcg_at_5
      value: 90.993
    - type: precision_at_1
      value: 84.04700000000001
    - type: precision_at_10
      value: 9.763
    - type: precision_at_100
      value: 0.9939999999999999
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.912
    - type: precision_at_3
      value: 31.490000000000002
    - type: precision_at_5
      value: 19.204
    - type: recall_at_1
      value: 84.04700000000001
    - type: recall_at_10
      value: 97.63
    - type: recall_at_100
      value: 99.423
    - type: recall_at_1000
      value: 99.909
    - type: recall_at_20
      value: 98.238
    - type: recall_at_3
      value: 94.47
    - type: recall_at_5
      value: 96.019
    task:
      type: Retrieval
  - dataset:
      config: go
      name: MTEB CodeSearchNetCCRetrieval (go)
      revision: 6e1effa2c03723c5fde48ee912b5ee08d4f211e8
      split: test
      type: CoIR-Retrieval/CodeSearchNet-ccr
    metrics:
    - type: main_score
      value: 91.75999999999999
    - type: map_at_1
      value: 84.28999999999999
    - type: map_at_10
      value: 89.689
    - type: map_at_100
      value: 89.773
    - type: map_at_1000
      value: 89.774
    - type: map_at_20
      value: 89.747
    - type: map_at_3
      value: 89.02600000000001
    - type: map_at_5
      value: 89.484
    - type: mrr_at_1
      value: 84.27727160797834
    - type: mrr_at_10
      value: 89.67515722532931
    - type: mrr_at_100
      value: 89.75855757010807
    - type: mrr_at_1000
      value: 89.7601837108608
    - type: mrr_at_20
      value: 89.73302201153147
    - type: mrr_at_3
      value: 89.01337929902301
    - type: mrr_at_5
      value: 89.47262579003495
    - type: nauc_map_at_1000_diff1
      value: 85.17428924238607
    - type: nauc_map_at_1000_max
      value: 48.685961107484225
    - type: nauc_map_at_1000_std
      value: -30.709156511297298
    - type: nauc_map_at_100_diff1
      value: 85.17188089036237
    - type: nauc_map_at_100_max
      value: 48.6913539422267
    - type: nauc_map_at_100_std
      value: -30.704177527491318
    - type: nauc_map_at_10_diff1
      value: 85.16218860131448
    - type: nauc_map_at_10_max
      value: 48.8187190573415
    - type: nauc_map_at_10_std
      value: -30.769000848055377
    - type: nauc_map_at_1_diff1
      value: 86.00618812234512
    - type: nauc_map_at_1_max
      value: 45.7845929261639
    - type: nauc_map_at_1_std
      value: -29.077490123642914
    - type: nauc_map_at_20_diff1
      value: 85.17186125725058
    - type: nauc_map_at_20_max
      value: 48.7405854117929
    - type: nauc_map_at_20_std
      value: -30.71428055810858
    - type: nauc_map_at_3_diff1
      value: 85.04460665149782
    - type: nauc_map_at_3_max
      value: 49.06920858786511
    - type: nauc_map_at_3_std
      value: -31.417225141010363
    - type: nauc_map_at_5_diff1
      value: 85.11442620241468
    - type: nauc_map_at_5_max
      value: 48.834954058956406
    - type: nauc_map_at_5_std
      value: -31.287907384502528
    - type: nauc_mrr_at_1000_diff1
      value: 85.21632400557363
    - type: nauc_mrr_at_1000_max
      value: 48.84146877454467
    - type: nauc_mrr_at_1000_std
      value: -30.7223552994054
    - type: nauc_mrr_at_100_diff1
      value: 85.2139127988098
    - type: nauc_mrr_at_100_max
      value: 48.84683704195986
    - type: nauc_mrr_at_100_std
      value: -30.717390859364478
    - type: nauc_mrr_at_10_diff1
      value: 85.20230661077468
    - type: nauc_mrr_at_10_max
      value: 48.957929042209614
    - type: nauc_mrr_at_10_std
      value: -30.78597710748554
    - type: nauc_mrr_at_1_diff1
      value: 86.0269378207981
    - type: nauc_mrr_at_1_max
      value: 45.94255127280741
    - type: nauc_mrr_at_1_std
      value: -29.011204207077935
    - type: nauc_mrr_at_20_diff1
      value: 85.21586630373561
    - type: nauc_mrr_at_20_max
      value: 48.88183732057669
    - type: nauc_mrr_at_20_std
      value: -30.73183934707418
    - type: nauc_mrr_at_3_diff1
      value: 85.08479102826571
    - type: nauc_mrr_at_3_max
      value: 49.23008482061558
    - type: nauc_mrr_at_3_std
      value: -31.465046952057303
    - type: nauc_mrr_at_5_diff1
      value: 85.1584637044273
    - type: nauc_mrr_at_5_max
      value: 49.00177049406493
    - type: nauc_mrr_at_5_std
      value: -31.307066011998224
    - type: nauc_ndcg_at_1000_diff1
      value: 85.09478158429043
    - type: nauc_ndcg_at_1000_max
      value: 49.25560935342975
    - type: nauc_ndcg_at_1000_std
      value: -30.70761801312305
    - type: nauc_ndcg_at_100_diff1
      value: 85.03251451319845
    - type: nauc_ndcg_at_100_max
      value: 49.42104394245281
    - type: nauc_ndcg_at_100_std
      value: -30.551562494744168
    - type: nauc_ndcg_at_10_diff1
      value: 85.00106483349344
    - type: nauc_ndcg_at_10_max
      value: 50.115231910648575
    - type: nauc_ndcg_at_10_std
      value: -30.87312388259668
    - type: nauc_ndcg_at_1_diff1
      value: 86.00618812234512
    - type: nauc_ndcg_at_1_max
      value: 45.7845929261639
    - type: nauc_ndcg_at_1_std
      value: -29.077490123642914
    - type: nauc_ndcg_at_20_diff1
      value: 85.05361380063081
    - type: nauc_ndcg_at_20_max
      value: 49.772465794293254
    - type: nauc_ndcg_at_20_std
      value: -30.672595686871666
    - type: nauc_ndcg_at_3_diff1
      value: 84.77305397981563
    - type: nauc_ndcg_at_3_max
      value: 50.52441698065767
    - type: nauc_ndcg_at_3_std
      value: -32.46848973997472
    - type: nauc_ndcg_at_5_diff1
      value: 84.88420002224352
    - type: nauc_ndcg_at_5_max
      value: 50.11133138392584
    - type: nauc_ndcg_at_5_std
      value: -32.3270235813713
    - type: nauc_precision_at_1000_diff1
      value: 91.83038946833551
    - type: nauc_precision_at_1000_max
      value: 88.46880842429545
    - type: nauc_precision_at_1000_std
      value: 21.769969322436477
    - type: nauc_precision_at_100_diff1
      value: 76.47557733881311
    - type: nauc_precision_at_100_max
      value: 84.28732698302338
    - type: nauc_precision_at_100_std
      value: 5.283523585432388
    - type: nauc_precision_at_10_diff1
      value: 83.18181354333525
    - type: nauc_precision_at_10_max
      value: 68.00168082360678
    - type: nauc_precision_at_10_std
      value: -29.843581018958474
    - type: nauc_precision_at_1_diff1
      value: 86.00618812234512
    - type: nauc_precision_at_1_max
      value: 45.7845929261639
    - type: nauc_precision_at_1_std
      value: -29.077490123642914
    - type: nauc_precision_at_20_diff1
      value: 83.4769114952918
    - type: nauc_precision_at_20_max
      value: 71.01630819186266
    - type: nauc_precision_at_20_std
      value: -24.608779769104114
    - type: nauc_precision_at_3_diff1
      value: 83.38151337361876
    - type: nauc_precision_at_3_max
      value: 58.48742413861129
    - type: nauc_precision_at_3_std
      value: -38.23005616508197
    - type: nauc_precision_at_5_diff1
      value: 83.16792264231816
    - type: nauc_precision_at_5_max
      value: 60.3244549312709
    - type: nauc_precision_at_5_std
      value: -40.87299770895208
    - type: nauc_recall_at_1000_diff1
      value: 91.83038946832791
    - type: nauc_recall_at_1000_max
      value: 88.46880842431686
    - type: nauc_recall_at_1000_std
      value: 21.769969322480403
    - type: nauc_recall_at_100_diff1
      value: 76.47557733881604
    - type: nauc_recall_at_100_max
      value: 84.28732698303843
    - type: nauc_recall_at_100_std
      value: 5.283523585434394
    - type: nauc_recall_at_10_diff1
      value: 83.18181354333507
    - type: nauc_recall_at_10_max
      value: 68.00168082360621
    - type: nauc_recall_at_10_std
      value: -29.843581018957572
    - type: nauc_recall_at_1_diff1
      value: 86.00618812234512
    - type: nauc_recall_at_1_max
      value: 45.7845929261639
    - type: nauc_recall_at_1_std
      value: -29.077490123642914
    - type: nauc_recall_at_20_diff1
      value: 83.47691149529236
    - type: nauc_recall_at_20_max
      value: 71.01630819186342
    - type: nauc_recall_at_20_std
      value: -24.608779769102323
    - type: nauc_recall_at_3_diff1
      value: 83.38151337361859
    - type: nauc_recall_at_3_max
      value: 58.48742413861102
    - type: nauc_recall_at_3_std
      value: -38.23005616508199
    - type: nauc_recall_at_5_diff1
      value: 83.16792264231778
    - type: nauc_recall_at_5_max
      value: 60.32445493127084
    - type: nauc_recall_at_5_std
      value: -40.8729977089511
    - type: ndcg_at_1
      value: 84.28999999999999
    - type: ndcg_at_10
      value: 91.75999999999999
    - type: ndcg_at_100
      value: 92.128
    - type: ndcg_at_1000
      value: 92.166
    - type: ndcg_at_20
      value: 91.965
    - type: ndcg_at_3
      value: 90.441
    - type: ndcg_at_5
      value: 91.264
    - type: precision_at_1
      value: 84.28999999999999
    - type: precision_at_10
      value: 9.802
    - type: precision_at_100
      value: 0.997
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.941
    - type: precision_at_3
      value: 31.499
    - type: precision_at_5
      value: 19.298000000000002
    - type: recall_at_1
      value: 84.28999999999999
    - type: recall_at_10
      value: 98.018
    - type: recall_at_100
      value: 99.655
    - type: recall_at_1000
      value: 99.938
    - type: recall_at_20
      value: 98.818
    - type: recall_at_3
      value: 94.49600000000001
    - type: recall_at_5
      value: 96.491
    task:
      type: Retrieval
  - dataset:
      config: ruby
      name: MTEB CodeSearchNetCCRetrieval (ruby)
      revision: 6e1effa2c03723c5fde48ee912b5ee08d4f211e8
      split: test
      type: CoIR-Retrieval/CodeSearchNet-ccr
    metrics:
    - type: main_score
      value: 89.25699999999999
    - type: map_at_1
      value: 80.967
    - type: map_at_10
      value: 86.978
    - type: map_at_100
      value: 87.113
    - type: map_at_1000
      value: 87.117
    - type: map_at_20
      value: 87.054
    - type: map_at_3
      value: 86.29400000000001
    - type: map_at_5
      value: 86.738
    - type: mrr_at_1
      value: 80.9674861221253
    - type: mrr_at_10
      value: 86.95775587528166
    - type: mrr_at_100
      value: 87.09311803235961
    - type: mrr_at_1000
      value: 87.09630839341808
    - type: mrr_at_20
      value: 87.03383454810835
    - type: mrr_at_3
      value: 86.25429553264607
    - type: mrr_at_5
      value: 86.71821305841928
    - type: nauc_map_at_1000_diff1
      value: 82.17808945938484
    - type: nauc_map_at_1000_max
      value: 60.75779539267633
    - type: nauc_map_at_1000_std
      value: -23.755896580375826
    - type: nauc_map_at_100_diff1
      value: 82.1794560801495
    - type: nauc_map_at_100_max
      value: 60.76366486911127
    - type: nauc_map_at_100_std
      value: -23.74131956559222
    - type: nauc_map_at_10_diff1
      value: 82.18782847323823
    - type: nauc_map_at_10_max
      value: 60.91576878869061
    - type: nauc_map_at_10_std
      value: -23.863677820347533
    - type: nauc_map_at_1_diff1
      value: 83.72039170136367
    - type: nauc_map_at_1_max
      value: 57.25380612858921
    - type: nauc_map_at_1_std
      value: -25.935791475913987
    - type: nauc_map_at_20_diff1
      value: 82.17796635750986
    - type: nauc_map_at_20_max
      value: 60.83301592914025
    - type: nauc_map_at_20_std
      value: -23.683955007198033
    - type: nauc_map_at_3_diff1
      value: 81.83929922286214
    - type: nauc_map_at_3_max
      value: 61.30090311509839
    - type: nauc_map_at_3_std
      value: -23.67250024837735
    - type: nauc_map_at_5_diff1
      value: 81.88267528957118
    - type: nauc_map_at_5_max
      value: 61.06118360132867
    - type: nauc_map_at_5_std
      value: -23.564202037953418
    - type: nauc_mrr_at_1000_diff1
      value: 82.24831373360206
    - type: nauc_mrr_at_1000_max
      value: 60.547458216811776
    - type: nauc_mrr_at_1000_std
      value: -24.62799535642754
    - type: nauc_mrr_at_100_diff1
      value: 82.24966010195052
    - type: nauc_mrr_at_100_max
      value: 60.553373399867574
    - type: nauc_mrr_at_100_std
      value: -24.613214185234447
    - type: nauc_mrr_at_10_diff1
      value: 82.25711255868444
    - type: nauc_mrr_at_10_max
      value: 60.70674951461077
    - type: nauc_mrr_at_10_std
      value: -24.723134285637112
    - type: nauc_mrr_at_1_diff1
      value: 83.72039170136367
    - type: nauc_mrr_at_1_max
      value: 56.98766026206557
    - type: nauc_mrr_at_1_std
      value: -26.606885807082385
    - type: nauc_mrr_at_20_diff1
      value: 82.24783610762223
    - type: nauc_mrr_at_20_max
      value: 60.62362881960727
    - type: nauc_mrr_at_20_std
      value: -24.551756212224145
    - type: nauc_mrr_at_3_diff1
      value: 81.93274024955835
    - type: nauc_mrr_at_3_max
      value: 61.09367721118326
    - type: nauc_mrr_at_3_std
      value: -24.713070452305637
    - type: nauc_mrr_at_5_diff1
      value: 81.95108632175992
    - type: nauc_mrr_at_5_max
      value: 60.85590424882734
    - type: nauc_mrr_at_5_std
      value: -24.40785466160273
    - type: nauc_ndcg_at_1000_diff1
      value: 82.13638313064818
    - type: nauc_ndcg_at_1000_max
      value: 61.36229533164094
    - type: nauc_ndcg_at_1000_std
      value: -23.263619782617027
    - type: nauc_ndcg_at_100_diff1
      value: 82.17708460929015
    - type: nauc_ndcg_at_100_max
      value: 61.55039283837963
    - type: nauc_ndcg_at_100_std
      value: -22.82475534090763
    - type: nauc_ndcg_at_10_diff1
      value: 82.19581572094847
    - type: nauc_ndcg_at_10_max
      value: 62.250754196853194
    - type: nauc_ndcg_at_10_std
      value: -23.522643567705575
    - type: nauc_ndcg_at_1_diff1
      value: 83.72039170136367
    - type: nauc_ndcg_at_1_max
      value: 57.25380612858921
    - type: nauc_ndcg_at_1_std
      value: -25.935791475913987
    - type: nauc_ndcg_at_20_diff1
      value: 82.14559730614958
    - type: nauc_ndcg_at_20_max
      value: 61.93079238970315
    - type: nauc_ndcg_at_20_std
      value: -22.719105026052123
    - type: nauc_ndcg_at_3_diff1
      value: 81.36414038291007
    - type: nauc_ndcg_at_3_max
      value: 62.97389940233761
    - type: nauc_ndcg_at_3_std
      value: -22.958679785157702
    - type: nauc_ndcg_at_5_diff1
      value: 81.41077106697202
    - type: nauc_ndcg_at_5_max
      value: 62.605999596646335
    - type: nauc_ndcg_at_5_std
      value: -22.684305831996262
    - type: nauc_precision_at_1000_diff1
      value: 100.0
    - type: nauc_precision_at_1000_max
      value: 100.0
    - type: nauc_precision_at_1000_std
      value: 55.42527628232039
    - type: nauc_precision_at_100_diff1
      value: 87.74666354712059
    - type: nauc_precision_at_100_max
      value: 83.08111330779417
    - type: nauc_precision_at_100_std
      value: 25.395348413223466
    - type: nauc_precision_at_10_diff1
      value: 83.35598753622322
    - type: nauc_precision_at_10_max
      value: 73.70169446342423
    - type: nauc_precision_at_10_std
      value: -21.615549803915652
    - type: nauc_precision_at_1_diff1
      value: 83.72039170136367
    - type: nauc_precision_at_1_max
      value: 57.25380612858921
    - type: nauc_precision_at_1_std
      value: -25.935791475913987
    - type: nauc_precision_at_20_diff1
      value: 83.03143013817835
    - type: nauc_precision_at_20_max
      value: 73.49113429650389
    - type: nauc_precision_at_20_std
      value: -9.826961687536869
    - type: nauc_precision_at_3_diff1
      value: 79.16945277065827
    - type: nauc_precision_at_3_max
      value: 71.31564419236823
    - type: nauc_precision_at_3_std
      value: -19.52745553766472
    - type: nauc_precision_at_5_diff1
      value: 78.5956097040408
    - type: nauc_precision_at_5_max
      value: 72.48547478245246
    - type: nauc_precision_at_5_std
      value: -17.07438252522886
    - type: nauc_recall_at_1000_diff1
      value: 100.0
    - type: nauc_recall_at_1000_max
      value: 100.0
    - type: nauc_recall_at_1000_std
      value: 55.42527628229722
    - type: nauc_recall_at_100_diff1
      value: 87.74666354712278
    - type: nauc_recall_at_100_max
      value: 83.08111330779381
    - type: nauc_recall_at_100_std
      value: 25.39534841322166
    - type: nauc_recall_at_10_diff1
      value: 83.35598753622315
    - type: nauc_recall_at_10_max
      value: 73.70169446342429
    - type: nauc_recall_at_10_std
      value: -21.615549803915744
    - type: nauc_recall_at_1_diff1
      value: 83.72039170136367
    - type: nauc_recall_at_1_max
      value: 57.25380612858921
    - type: nauc_recall_at_1_std
      value: -25.935791475913987
    - type: nauc_recall_at_20_diff1
      value: 83.03143013817795
    - type: nauc_recall_at_20_max
      value: 73.49113429650306
    - type: nauc_recall_at_20_std
      value: -9.826961687536835
    - type: nauc_recall_at_3_diff1
      value: 79.16945277065781
    - type: nauc_recall_at_3_max
      value: 71.31564419236814
    - type: nauc_recall_at_3_std
      value: -19.527455537664128
    - type: nauc_recall_at_5_diff1
      value: 78.59560970404043
    - type: nauc_recall_at_5_max
      value: 72.4854747824523
    - type: nauc_recall_at_5_std
      value: -17.074382525229172
    - type: ndcg_at_1
      value: 80.967
    - type: ndcg_at_10
      value: 89.25699999999999
    - type: ndcg_at_100
      value: 89.91799999999999
    - type: ndcg_at_1000
      value: 90.007
    - type: ndcg_at_20
      value: 89.524
    - type: ndcg_at_3
      value: 87.909
    - type: ndcg_at_5
      value: 88.701
    - type: precision_at_1
      value: 80.967
    - type: precision_at_10
      value: 9.611
    - type: precision_at_100
      value: 0.992
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.857
    - type: precision_at_3
      value: 30.849
    - type: precision_at_5
      value: 18.89
    - type: recall_at_1
      value: 80.967
    - type: recall_at_10
      value: 96.114
    - type: recall_at_100
      value: 99.207
    - type: recall_at_1000
      value: 99.921
    - type: recall_at_20
      value: 97.14500000000001
    - type: recall_at_3
      value: 92.54599999999999
    - type: recall_at_5
      value: 94.44900000000001
    task:
      type: Retrieval
  - dataset:
      config: java
      name: MTEB CodeSearchNetCCRetrieval (java)
      revision: 6e1effa2c03723c5fde48ee912b5ee08d4f211e8
      split: test
      type: CoIR-Retrieval/CodeSearchNet-ccr
    metrics:
    - type: main_score
      value: 90.187
    - type: map_at_1
      value: 83.04899999999999
    - type: map_at_10
      value: 88.2
    - type: map_at_100
      value: 88.294
    - type: map_at_1000
      value: 88.297
    - type: map_at_20
      value: 88.268
    - type: map_at_3
      value: 87.581
    - type: map_at_5
      value: 87.997
    - type: mrr_at_1
      value: 83.03057964399817
    - type: mrr_at_10
      value: 88.19186353987806
    - type: mrr_at_100
      value: 88.2852857857312
    - type: mrr_at_1000
      value: 88.28810205243452
    - type: mrr_at_20
      value: 88.25951626896247
    - type: mrr_at_3
      value: 87.5749277346721
    - type: mrr_at_5
      value: 87.9875247223489
    - type: nauc_map_at_1000_diff1
      value: 84.10730394433992
    - type: nauc_map_at_1000_max
      value: 61.52233212127969
    - type: nauc_map_at_1000_std
      value: -13.441699805696281
    - type: nauc_map_at_100_diff1
      value: 84.10552079587292
    - type: nauc_map_at_100_max
      value: 61.52588029468992
    - type: nauc_map_at_100_std
      value: -13.438768702376994
    - type: nauc_map_at_10_diff1
      value: 84.06622800697097
    - type: nauc_map_at_10_max
      value: 61.62852938331853
    - type: nauc_map_at_10_std
      value: -13.540123911279858
    - type: nauc_map_at_1_diff1
      value: 84.58928110942394
    - type: nauc_map_at_1_max
      value: 56.15285789003811
    - type: nauc_map_at_1_std
      value: -17.221403594342735
    - type: nauc_map_at_20_diff1
      value: 84.09390965699994
    - type: nauc_map_at_20_max
      value: 61.569980648560886
    - type: nauc_map_at_20_std
      value: -13.445035300847309
    - type: nauc_map_at_3_diff1
      value: 83.88744798135446
    - type: nauc_map_at_3_max
      value: 61.88368527363093
    - type: nauc_map_at_3_std
      value: -14.711176741863827
    - type: nauc_map_at_5_diff1
      value: 83.98674004934382
    - type: nauc_map_at_5_max
      value: 61.74583336524621
    - type: nauc_map_at_5_std
      value: -13.965194914228382
    - type: nauc_mrr_at_1000_diff1
      value: 84.07413270132193
    - type: nauc_mrr_at_1000_max
      value: 61.703650435638856
    - type: nauc_mrr_at_1000_std
      value: -13.350251134092783
    - type: nauc_mrr_at_100_diff1
      value: 84.07235854433452
    - type: nauc_mrr_at_100_max
      value: 61.707169972354
    - type: nauc_mrr_at_100_std
      value: -13.347338815823864
    - type: nauc_mrr_at_10_diff1
      value: 84.03062092695845
    - type: nauc_mrr_at_10_max
      value: 61.80570291957679
    - type: nauc_mrr_at_10_std
      value: -13.438890580569844
    - type: nauc_mrr_at_1_diff1
      value: 84.61937189362341
    - type: nauc_mrr_at_1_max
      value: 56.41943088748145
    - type: nauc_mrr_at_1_std
      value: -17.177446751086197
    - type: nauc_mrr_at_20_diff1
      value: 84.06068844396994
    - type: nauc_mrr_at_20_max
      value: 61.750926871721475
    - type: nauc_mrr_at_20_std
      value: -13.35357387841223
    - type: nauc_mrr_at_3_diff1
      value: 83.85345674418377
    - type: nauc_mrr_at_3_max
      value: 62.043812033960535
    - type: nauc_mrr_at_3_std
      value: -14.6395621624969
    - type: nauc_mrr_at_5_diff1
      value: 83.95764939870173
    - type: nauc_mrr_at_5_max
      value: 61.92516669938368
    - type: nauc_mrr_at_5_std
      value: -13.886532596685806
    - type: nauc_ndcg_at_1000_diff1
      value: 84.42408974340631
    - type: nauc_ndcg_at_1000_max
      value: 63.32980075889167
    - type: nauc_ndcg_at_1000_std
      value: -9.859850962580813
    - type: nauc_ndcg_at_100_diff1
      value: 84.3765359347518
    - type: nauc_ndcg_at_100_max
      value: 63.39891417852467
    - type: nauc_ndcg_at_100_std
      value: -9.822808442396664
    - type: nauc_ndcg_at_10_diff1
      value: 84.14419181720488
    - type: nauc_ndcg_at_10_max
      value: 63.950601713958136
    - type: nauc_ndcg_at_10_std
      value: -10.490521537540003
    - type: nauc_ndcg_at_1_diff1
      value: 84.58928110942394
    - type: nauc_ndcg_at_1_max
      value: 56.15285789003811
    - type: nauc_ndcg_at_1_std
      value: -17.221403594342735
    - type: nauc_ndcg_at_20_diff1
      value: 84.26722898082026
    - type: nauc_ndcg_at_20_max
      value: 63.749766448071746
    - type: nauc_ndcg_at_20_std
      value: -10.004576105283306
    - type: nauc_ndcg_at_3_diff1
      value: 83.72365997941274
    - type: nauc_ndcg_at_3_max
      value: 64.3195097894205
    - type: nauc_ndcg_at_3_std
      value: -13.41750319915828
    - type: nauc_ndcg_at_5_diff1
      value: 83.91163247137122
    - type: nauc_ndcg_at_5_max
      value: 64.19792463538528
    - type: nauc_ndcg_at_5_std
      value: -11.76492800063572
    - type: nauc_precision_at_1000_diff1
      value: 97.13217959743038
    - type: nauc_precision_at_1000_max
      value: 98.46911971679215
    - type: nauc_precision_at_1000_std
      value: 90.2347995052506
    - type: nauc_precision_at_100_diff1
      value: 91.63251470011757
    - type: nauc_precision_at_100_max
      value: 90.17514956130655
    - type: nauc_precision_at_100_std
      value: 61.0806660186732
    - type: nauc_precision_at_10_diff1
      value: 85.28726061856946
    - type: nauc_precision_at_10_max
      value: 83.23195279280301
    - type: nauc_precision_at_10_std
      value: 17.658421447762464
    - type: nauc_precision_at_1_diff1
      value: 84.58928110942394
    - type: nauc_precision_at_1_max
      value: 56.15285789003811
    - type: nauc_precision_at_1_std
      value: -17.221403594342735
    - type: nauc_precision_at_20_diff1
      value: 87.21693331269776
    - type: nauc_precision_at_20_max
      value: 86.97422806229869
    - type: nauc_precision_at_20_std
      value: 32.947726980858015
    - type: nauc_precision_at_3_diff1
      value: 83.00721034174204
    - type: nauc_precision_at_3_max
      value: 76.03478845540039
    - type: nauc_precision_at_3_std
      value: -7.075307314296672
    - type: nauc_precision_at_5_diff1
      value: 83.59241003022704
    - type: nauc_precision_at_5_max
      value: 79.35871830248998
    - type: nauc_precision_at_5_std
      value: 2.590360587580312
    - type: nauc_recall_at_1000_diff1
      value: 97.13217959743086
    - type: nauc_recall_at_1000_max
      value: 98.46911971679087
    - type: nauc_recall_at_1000_std
      value: 90.23479950524963
    - type: nauc_recall_at_100_diff1
      value: 91.6325147001174
    - type: nauc_recall_at_100_max
      value: 90.17514956130668
    - type: nauc_recall_at_100_std
      value: 61.08066601867284
    - type: nauc_recall_at_10_diff1
      value: 85.28726061856851
    - type: nauc_recall_at_10_max
      value: 83.23195279280242
    - type: nauc_recall_at_10_std
      value: 17.65842144776203
    - type: nauc_recall_at_1_diff1
      value: 84.58928110942394
    - type: nauc_recall_at_1_max
      value: 56.15285789003811
    - type: nauc_recall_at_1_std
      value: -17.221403594342735
    - type: nauc_recall_at_20_diff1
      value: 87.21693331269785
    - type: nauc_recall_at_20_max
      value: 86.97422806229821
    - type: nauc_recall_at_20_std
      value: 32.94772698085802
    - type: nauc_recall_at_3_diff1
      value: 83.007210341742
    - type: nauc_recall_at_3_max
      value: 76.03478845540012
    - type: nauc_recall_at_3_std
      value: -7.075307314297068
    - type: nauc_recall_at_5_diff1
      value: 83.59241003022677
    - type: nauc_recall_at_5_max
      value: 79.35871830248969
    - type: nauc_recall_at_5_std
      value: 2.5903605875805535
    - type: ndcg_at_1
      value: 83.04899999999999
    - type: ndcg_at_10
      value: 90.187
    - type: ndcg_at_100
      value: 90.61399999999999
    - type: ndcg_at_1000
      value: 90.686
    - type: ndcg_at_20
      value: 90.429
    - type: ndcg_at_3
      value: 88.957
    - type: ndcg_at_5
      value: 89.7
    - type: precision_at_1
      value: 83.04899999999999
    - type: precision_at_10
      value: 9.619
    - type: precision_at_100
      value: 0.9809999999999999
    - type: precision_at_1000
      value: 0.099
    - type: precision_at_20
      value: 4.857
    - type: precision_at_3
      value: 30.969
    - type: precision_at_5
      value: 18.939
    - type: recall_at_1
      value: 83.04899999999999
    - type: recall_at_10
      value: 96.194
    - type: recall_at_100
      value: 98.129
    - type: recall_at_1000
      value: 98.695
    - type: recall_at_20
      value: 97.134
    - type: recall_at_3
      value: 92.907
    - type: recall_at_5
      value: 94.696
    task:
      type: Retrieval
  - dataset:
      config: php
      name: MTEB CodeSearchNetCCRetrieval (php)
      revision: 6e1effa2c03723c5fde48ee912b5ee08d4f211e8
      split: test
      type: CoIR-Retrieval/CodeSearchNet-ccr
    metrics:
    - type: main_score
      value: 83.38799999999999
    - type: map_at_1
      value: 73.25500000000001
    - type: map_at_10
      value: 80.387
    - type: map_at_100
      value: 80.64099999999999
    - type: map_at_1000
      value: 80.648
    - type: map_at_20
      value: 80.55799999999999
    - type: map_at_3
      value: 79.256
    - type: map_at_5
      value: 79.97800000000001
    - type: mrr_at_1
      value: 73.24818039103754
    - type: mrr_at_10
      value: 80.37402789953873
    - type: mrr_at_100
      value: 80.62749361355517
    - type: mrr_at_1000
      value: 80.63492699285514
    - type: mrr_at_20
      value: 80.5450937498758
    - type: mrr_at_3
      value: 79.25170068027252
    - type: mrr_at_5
      value: 79.96063460349235
    - type: nauc_map_at_1000_diff1
      value: 77.55679733254976
    - type: nauc_map_at_1000_max
      value: 47.30416473970588
    - type: nauc_map_at_1000_std
      value: -26.244493881093646
    - type: nauc_map_at_100_diff1
      value: 77.55521791705124
    - type: nauc_map_at_100_max
      value: 47.30912923045687
    - type: nauc_map_at_100_std
      value: -26.243222353374808
    - type: nauc_map_at_10_diff1
      value: 77.46319200229681
    - type: nauc_map_at_10_max
      value: 47.42459632571874
    - type: nauc_map_at_10_std
      value: -26.359798504203013
    - type: nauc_map_at_1_diff1
      value: 79.527051162636
    - type: nauc_map_at_1_max
      value: 44.89631724552463
    - type: nauc_map_at_1_std
      value: -25.451766234360097
    - type: nauc_map_at_20_diff1
      value: 77.52788532848108
    - type: nauc_map_at_20_max
      value: 47.33830573278101
    - type: nauc_map_at_20_std
      value: -26.257482321628622
    - type: nauc_map_at_3_diff1
      value: 77.41062263238054
    - type: nauc_map_at_3_max
      value: 47.499244399918155
    - type: nauc_map_at_3_std
      value: -26.523866758341992
    - type: nauc_map_at_5_diff1
      value: 77.35129682203655
    - type: nauc_map_at_5_max
      value: 47.47681231525524
    - type: nauc_map_at_5_std
      value: -26.5173636683384
    - type: nauc_mrr_at_1000_diff1
      value: 77.57769206330354
    - type: nauc_mrr_at_1000_max
      value: 47.34367340151292
    - type: nauc_mrr_at_1000_std
      value: -26.1670476224202
    - type: nauc_mrr_at_100_diff1
      value: 77.57602878422465
    - type: nauc_mrr_at_100_max
      value: 47.34876516670401
    - type: nauc_mrr_at_100_std
      value: -26.16596878042115
    - type: nauc_mrr_at_10_diff1
      value: 77.48926507168112
    - type: nauc_mrr_at_10_max
      value: 47.46252680868696
    - type: nauc_mrr_at_10_std
      value: -26.282201461393317
    - type: nauc_mrr_at_1_diff1
      value: 79.53993768139632
    - type: nauc_mrr_at_1_max
      value: 44.95304522643992
    - type: nauc_mrr_at_1_std
      value: -25.32633885677888
    - type: nauc_mrr_at_20_diff1
      value: 77.54890591729169
    - type: nauc_mrr_at_20_max
      value: 47.37631632390287
    - type: nauc_mrr_at_20_std
      value: -26.183017296393896
    - type: nauc_mrr_at_3_diff1
      value: 77.4534326297116
    - type: nauc_mrr_at_3_max
      value: 47.54929766473431
    - type: nauc_mrr_at_3_std
      value: -26.432202942463785
    - type: nauc_mrr_at_5_diff1
      value: 77.38739995097214
    - type: nauc_mrr_at_5_max
      value: 47.50110390682936
    - type: nauc_mrr_at_5_std
      value: -26.463416200561753
    - type: nauc_ndcg_at_1000_diff1
      value: 77.20842500889081
    - type: nauc_ndcg_at_1000_max
      value: 47.70813895735421
    - type: nauc_ndcg_at_1000_std
      value: -26.023069702012446
    - type: nauc_ndcg_at_100_diff1
      value: 77.16015320575458
    - type: nauc_ndcg_at_100_max
      value: 47.88272658215937
    - type: nauc_ndcg_at_100_std
      value: -25.893271327002005
    - type: nauc_ndcg_at_10_diff1
      value: 76.68338764268012
    - type: nauc_ndcg_at_10_max
      value: 48.4217483255242
    - type: nauc_ndcg_at_10_std
      value: -26.36940839993425
    - type: nauc_ndcg_at_1_diff1
      value: 79.527051162636
    - type: nauc_ndcg_at_1_max
      value: 44.89631724552463
    - type: nauc_ndcg_at_1_std
      value: -25.451766234360097
    - type: nauc_ndcg_at_20_diff1
      value: 76.9371111655833
    - type: nauc_ndcg_at_20_max
      value: 48.10138804970404
    - type: nauc_ndcg_at_20_std
      value: -25.97419280800229
    - type: nauc_ndcg_at_3_diff1
      value: 76.61248461087024
    - type: nauc_ndcg_at_3_max
      value: 48.51663194946113
    - type: nauc_ndcg_at_3_std
      value: -26.789177280257793
    - type: nauc_ndcg_at_5_diff1
      value: 76.42970850899898
    - type: nauc_ndcg_at_5_max
      value: 48.52719429311673
    - type: nauc_ndcg_at_5_std
      value: -26.78716430555766
    - type: nauc_precision_at_1000_diff1
      value: 70.03887145024319
    - type: nauc_precision_at_1000_max
      value: 42.00416767646647
    - type: nauc_precision_at_1000_std
      value: -2.156778197450252
    - type: nauc_precision_at_100_diff1
      value: 72.9743402702392
    - type: nauc_precision_at_100_max
      value: 56.4734539374456
    - type: nauc_precision_at_100_std
      value: -12.25156037378497
    - type: nauc_precision_at_10_diff1
      value: 71.50550227914846
    - type: nauc_precision_at_10_max
      value: 55.096376602923144
    - type: nauc_precision_at_10_std
      value: -25.929290375863356
    - type: nauc_precision_at_1_diff1
      value: 79.527051162636
    - type: nauc_precision_at_1_max
      value: 44.89631724552463
    - type: nauc_precision_at_1_std
      value: -25.451766234360097
    - type: nauc_precision_at_20_diff1
      value: 71.99203963125369
    - type: nauc_precision_at_20_max
      value: 54.66712448432459
    - type: nauc_precision_at_20_std
      value: -21.478852653786358
    - type: nauc_precision_at_3_diff1
      value: 73.33785677931962
    - type: nauc_precision_at_3_max
      value: 52.70785427475565
    - type: nauc_precision_at_3_std
      value: -27.81147380600097
    - type: nauc_precision_at_5_diff1
      value: 71.76130552913553
    - type: nauc_precision_at_5_max
      value: 53.78865964066962
    - type: nauc_precision_at_5_std
      value: -28.044309543143616
    - type: nauc_recall_at_1000_diff1
      value: 70.03887145024234
    - type: nauc_recall_at_1000_max
      value: 42.00416767647104
    - type: nauc_recall_at_1000_std
      value: -2.15677819744854
    - type: nauc_recall_at_100_diff1
      value: 72.974340270239
    - type: nauc_recall_at_100_max
      value: 56.473453937445306
    - type: nauc_recall_at_100_std
      value: -12.25156037378462
    - type: nauc_recall_at_10_diff1
      value: 71.50550227914843
    - type: nauc_recall_at_10_max
      value: 55.096376602923
    - type: nauc_recall_at_10_std
      value: -25.929290375863673
    - type: nauc_recall_at_1_diff1
      value: 79.527051162636
    - type: nauc_recall_at_1_max
      value: 44.89631724552463
    - type: nauc_recall_at_1_std
      value: -25.451766234360097
    - type: nauc_recall_at_20_diff1
      value: 71.992039631253
    - type: nauc_recall_at_20_max
      value: 54.66712448432437
    - type: nauc_recall_at_20_std
      value: -21.478852653786607
    - type: nauc_recall_at_3_diff1
      value: 73.33785677931951
    - type: nauc_recall_at_3_max
      value: 52.707854274755626
    - type: nauc_recall_at_3_std
      value: -27.811473806001057
    - type: nauc_recall_at_5_diff1
      value: 71.76130552913536
    - type: nauc_recall_at_5_max
      value: 53.788659640669344
    - type: nauc_recall_at_5_std
      value: -28.044309543143715
    - type: ndcg_at_1
      value: 73.25500000000001
    - type: ndcg_at_10
      value: 83.38799999999999
    - type: ndcg_at_100
      value: 84.571
    - type: ndcg_at_1000
      value: 84.768
    - type: ndcg_at_20
      value: 83.99799999999999
    - type: ndcg_at_3
      value: 81.118
    - type: ndcg_at_5
      value: 82.40899999999999
    - type: precision_at_1
      value: 73.25500000000001
    - type: precision_at_10
      value: 9.259
    - type: precision_at_100
      value: 0.98
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.749
    - type: precision_at_3
      value: 28.826
    - type: precision_at_5
      value: 17.918
    - type: recall_at_1
      value: 73.25500000000001
    - type: recall_at_10
      value: 92.586
    - type: recall_at_100
      value: 98.016
    - type: recall_at_1000
      value: 99.572
    - type: recall_at_20
      value: 94.976
    - type: recall_at_3
      value: 86.478
    - type: recall_at_5
      value: 89.589
    task:
      type: Retrieval
  - dataset:
      config: python
      name: MTEB CodeSearchNetRetrieval (python)
      revision: fdc6a9e39575768c27eb8a2a5f702bf846eb4759
      split: test
      type: code-search-net/code_search_net
    metrics:
    - type: main_score
      value: 96.688
    - type: map_at_1
      value: 92.7
    - type: map_at_10
      value: 95.643
    - type: map_at_100
      value: 95.649
    - type: map_at_1000
      value: 95.649
    - type: map_at_20
      value: 95.643
    - type: map_at_3
      value: 95.417
    - type: map_at_5
      value: 95.562
    - type: mrr_at_1
      value: 92.7
    - type: mrr_at_10
      value: 95.64289682539683
    - type: mrr_at_100
      value: 95.64912393162395
    - type: mrr_at_1000
      value: 95.64912393162395
    - type: mrr_at_20
      value: 95.64289682539683
    - type: mrr_at_3
      value: 95.41666666666669
    - type: mrr_at_5
      value: 95.56166666666668
    - type: nauc_map_at_1000_diff1
      value: 93.2086804630001
    - type: nauc_map_at_1000_max
      value: 71.36336949003405
    - type: nauc_map_at_1000_std
      value: 7.542855305507389
    - type: nauc_map_at_100_diff1
      value: 93.2086804630001
    - type: nauc_map_at_100_max
      value: 71.36336949003405
    - type: nauc_map_at_100_std
      value: 7.542855305507389
    - type: nauc_map_at_10_diff1
      value: 93.21838651458631
    - type: nauc_map_at_10_max
      value: 71.40429653101195
    - type: nauc_map_at_10_std
      value: 7.674993664044225
    - type: nauc_map_at_1_diff1
      value: 92.97225739610897
    - type: nauc_map_at_1_max
      value: 69.06680480411342
    - type: nauc_map_at_1_std
      value: 8.97062021155494
    - type: nauc_map_at_20_diff1
      value: 93.21838651458631
    - type: nauc_map_at_20_max
      value: 71.40429653101195
    - type: nauc_map_at_20_std
      value: 7.674993664044225
    - type: nauc_map_at_3_diff1
      value: 93.22689075630251
    - type: nauc_map_at_3_max
      value: 71.83278159748735
    - type: nauc_map_at_3_std
      value: 5.838383838384187
    - type: nauc_map_at_5_diff1
      value: 93.07924446534143
    - type: nauc_map_at_5_max
      value: 71.48845769375428
    - type: nauc_map_at_5_std
      value: 6.123107648366922
    - type: nauc_mrr_at_1000_diff1
      value: 93.2086804630001
    - type: nauc_mrr_at_1000_max
      value: 71.36336949003405
    - type: nauc_mrr_at_1000_std
      value: 7.542855305507389
    - type: nauc_mrr_at_100_diff1
      value: 93.2086804630001
    - type: nauc_mrr_at_100_max
      value: 71.36336949003405
    - type: nauc_mrr_at_100_std
      value: 7.542855305507389
    - type: nauc_mrr_at_10_diff1
      value: 93.21838651458631
    - type: nauc_mrr_at_10_max
      value: 71.40429653101195
    - type: nauc_mrr_at_10_std
      value: 7.674993664044225
    - type: nauc_mrr_at_1_diff1
      value: 92.97225739610897
    - type: nauc_mrr_at_1_max
      value: 69.06680480411342
    - type: nauc_mrr_at_1_std
      value: 8.97062021155494
    - type: nauc_mrr_at_20_diff1
      value: 93.21838651458631
    - type: nauc_mrr_at_20_max
      value: 71.40429653101195
    - type: nauc_mrr_at_20_std
      value: 7.674993664044225
    - type: nauc_mrr_at_3_diff1
      value: 93.22689075630251
    - type: nauc_mrr_at_3_max
      value: 71.83278159748735
    - type: nauc_mrr_at_3_std
      value: 5.838383838384187
    - type: nauc_mrr_at_5_diff1
      value: 93.07924446534143
    - type: nauc_mrr_at_5_max
      value: 71.48845769375428
    - type: nauc_mrr_at_5_std
      value: 6.123107648366922
    - type: nauc_ndcg_at_1000_diff1
      value: 93.23304542841751
    - type: nauc_ndcg_at_1000_max
      value: 71.63083509229389
    - type: nauc_ndcg_at_1000_std
      value: 7.571755253398231
    - type: nauc_ndcg_at_100_diff1
      value: 93.23304542841751
    - type: nauc_ndcg_at_100_max
      value: 71.63083509229389
    - type: nauc_ndcg_at_100_std
      value: 7.571755253398231
    - type: nauc_ndcg_at_10_diff1
      value: 93.31367660323025
    - type: nauc_ndcg_at_10_max
      value: 71.96886589636927
    - type: nauc_ndcg_at_10_std
      value: 8.673077551486251
    - type: nauc_ndcg_at_1_diff1
      value: 92.97225739610897
    - type: nauc_ndcg_at_1_max
      value: 69.06680480411342
    - type: nauc_ndcg_at_1_std
      value: 8.97062021155494
    - type: nauc_ndcg_at_20_diff1
      value: 93.31367660323025
    - type: nauc_ndcg_at_20_max
      value: 71.96886589636927
    - type: nauc_ndcg_at_20_std
      value: 8.673077551486251
    - type: nauc_ndcg_at_3_diff1
      value: 93.23894522522178
    - type: nauc_ndcg_at_3_max
      value: 72.902496295325
    - type: nauc_ndcg_at_3_std
      value: 3.6178051090847845
    - type: nauc_ndcg_at_5_diff1
      value: 92.90972948390606
    - type: nauc_ndcg_at_5_max
      value: 72.19526067100512
    - type: nauc_ndcg_at_5_std
      value: 4.041582233975721
    - type: nauc_precision_at_1000_diff1
      value: .nan
    - type: nauc_precision_at_1000_max
      value: .nan
    - type: nauc_precision_at_1000_std
      value: .nan
    - type: nauc_precision_at_100_diff1
      value: 100.0
    - type: nauc_precision_at_100_max
      value: 100.0
    - type: nauc_precision_at_100_std
      value: 100.0
    - type: nauc_precision_at_10_diff1
      value: 100.0
    - type: nauc_precision_at_10_max
      value: 100.0
    - type: nauc_precision_at_10_std
      value: 100.0
    - type: nauc_precision_at_1_diff1
      value: 92.97225739610897
    - type: nauc_precision_at_1_max
      value: 69.06680480411342
    - type: nauc_precision_at_1_std
      value: 8.97062021155494
    - type: nauc_precision_at_20_diff1
      value: 100.0
    - type: nauc_precision_at_20_max
      value: 100.0
    - type: nauc_precision_at_20_std
      value: 100.0
    - type: nauc_precision_at_3_diff1
      value: 93.2306255835672
    - type: nauc_precision_at_3_max
      value: 82.46298519407813
    - type: nauc_precision_at_3_std
      value: -17.583700146723764
    - type: nauc_precision_at_5_diff1
      value: 89.78758169934665
    - type: nauc_precision_at_5_max
      value: 82.09033613445358
    - type: nauc_precision_at_5_std
      value: -29.580999066296027
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: .nan
    - type: nauc_recall_at_100_max
      value: .nan
    - type: nauc_recall_at_100_std
      value: .nan
    - type: nauc_recall_at_10_diff1
      value: 100.0
    - type: nauc_recall_at_10_max
      value: 100.0
    - type: nauc_recall_at_10_std
      value: 100.0
    - type: nauc_recall_at_1_diff1
      value: 92.97225739610897
    - type: nauc_recall_at_1_max
      value: 69.06680480411342
    - type: nauc_recall_at_1_std
      value: 8.97062021155494
    - type: nauc_recall_at_20_diff1
      value: 100.0
    - type: nauc_recall_at_20_max
      value: 100.0
    - type: nauc_recall_at_20_std
      value: 100.0
    - type: nauc_recall_at_3_diff1
      value: 93.2306255835672
    - type: nauc_recall_at_3_max
      value: 82.4629851940786
    - type: nauc_recall_at_3_std
      value: -17.5837001467247
    - type: nauc_recall_at_5_diff1
      value: 89.78758169934649
    - type: nauc_recall_at_5_max
      value: 82.09033613445361
    - type: nauc_recall_at_5_std
      value: -29.580999066293263
    - type: ndcg_at_1
      value: 92.7
    - type: ndcg_at_10
      value: 96.688
    - type: ndcg_at_100
      value: 96.72800000000001
    - type: ndcg_at_1000
      value: 96.72800000000001
    - type: ndcg_at_20
      value: 96.688
    - type: ndcg_at_3
      value: 96.23899999999999
    - type: ndcg_at_5
      value: 96.493
    - type: precision_at_1
      value: 92.7
    - type: precision_at_10
      value: 9.98
    - type: precision_at_100
      value: 1.0
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.99
    - type: precision_at_3
      value: 32.867000000000004
    - type: precision_at_5
      value: 19.84
    - type: recall_at_1
      value: 92.7
    - type: recall_at_10
      value: 99.8
    - type: recall_at_100
      value: 100.0
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 99.8
    - type: recall_at_3
      value: 98.6
    - type: recall_at_5
      value: 99.2
    task:
      type: Retrieval
  - dataset:
      config: javascript
      name: MTEB CodeSearchNetRetrieval (javascript)
      revision: fdc6a9e39575768c27eb8a2a5f702bf846eb4759
      split: test
      type: code-search-net/code_search_net
    metrics:
    - type: main_score
      value: 89.97800000000001
    - type: map_at_1
      value: 85.0
    - type: map_at_10
      value: 88.427
    - type: map_at_100
      value: 88.627
    - type: map_at_1000
      value: 88.631
    - type: map_at_20
      value: 88.588
    - type: map_at_3
      value: 87.767
    - type: map_at_5
      value: 88.19200000000001
    - type: mrr_at_1
      value: 85.0
    - type: mrr_at_10
      value: 88.42742063492065
    - type: mrr_at_100
      value: 88.62702226900807
    - type: mrr_at_1000
      value: 88.63072320739236
    - type: mrr_at_20
      value: 88.58809151632684
    - type: mrr_at_3
      value: 87.76666666666667
    - type: mrr_at_5
      value: 88.1916666666667
    - type: nauc_map_at_1000_diff1
      value: 90.72581837961151
    - type: nauc_map_at_1000_max
      value: 62.96996529181109
    - type: nauc_map_at_1000_std
      value: 5.4686257905510445
    - type: nauc_map_at_100_diff1
      value: 90.7251318399043
    - type: nauc_map_at_100_max
      value: 62.98186646957673
    - type: nauc_map_at_100_std
      value: 5.499359385640601
    - type: nauc_map_at_10_diff1
      value: 90.7954839398183
    - type: nauc_map_at_10_max
      value: 63.22150406130067
    - type: nauc_map_at_10_std
      value: 5.413600198301859
    - type: nauc_map_at_1_diff1
      value: 91.52066384640422
    - type: nauc_map_at_1_max
      value: 57.639765701269155
    - type: nauc_map_at_1_std
      value: 1.4959323136999483
    - type: nauc_map_at_20_diff1
      value: 90.71434972102585
    - type: nauc_map_at_20_max
      value: 63.07611119431469
    - type: nauc_map_at_20_std
      value: 5.640274998494253
    - type: nauc_map_at_3_diff1
      value: 90.33141255622978
    - type: nauc_map_at_3_max
      value: 62.34164710561914
    - type: nauc_map_at_3_std
      value: 3.5068574882828925
    - type: nauc_map_at_5_diff1
      value: 90.79552786535547
    - type: nauc_map_at_5_max
      value: 63.119944203478106
    - type: nauc_map_at_5_std
      value: 4.953420352568366
    - type: nauc_mrr_at_1000_diff1
      value: 90.72581837961151
    - type: nauc_mrr_at_1000_max
      value: 62.96996529181109
    - type: nauc_mrr_at_1000_std
      value: 5.4686257905510445
    - type: nauc_mrr_at_100_diff1
      value: 90.7251318399043
    - type: nauc_mrr_at_100_max
      value: 62.98186646957673
    - type: nauc_mrr_at_100_std
      value: 5.499359385640601
    - type: nauc_mrr_at_10_diff1
      value: 90.7954839398183
    - type: nauc_mrr_at_10_max
      value: 63.22150406130067
    - type: nauc_mrr_at_10_std
      value: 5.413600198301859
    - type: nauc_mrr_at_1_diff1
      value: 91.52066384640422
    - type: nauc_mrr_at_1_max
      value: 57.639765701269155
    - type: nauc_mrr_at_1_std
      value: 1.4959323136999483
    - type: nauc_mrr_at_20_diff1
      value: 90.71434972102585
    - type: nauc_mrr_at_20_max
      value: 63.07611119431469
    - type: nauc_mrr_at_20_std
      value: 5.640274998494253
    - type: nauc_mrr_at_3_diff1
      value: 90.33141255622978
    - type: nauc_mrr_at_3_max
      value: 62.34164710561914
    - type: nauc_mrr_at_3_std
      value: 3.5068574882828925
    - type: nauc_mrr_at_5_diff1
      value: 90.79552786535547
    - type: nauc_mrr_at_5_max
      value: 63.119944203478106
    - type: nauc_mrr_at_5_std
      value: 4.953420352568366
    - type: nauc_ndcg_at_1000_diff1
      value: 90.6732163208453
    - type: nauc_ndcg_at_1000_max
      value: 64.27542517763368
    - type: nauc_ndcg_at_1000_std
      value: 7.423716824052952
    - type: nauc_ndcg_at_100_diff1
      value: 90.65475476530615
    - type: nauc_ndcg_at_100_max
      value: 64.89222263529967
    - type: nauc_ndcg_at_100_std
      value: 9.036733154889918
    - type: nauc_ndcg_at_10_diff1
      value: 90.88391240350151
    - type: nauc_ndcg_at_10_max
      value: 66.10798661293683
    - type: nauc_ndcg_at_10_std
      value: 8.662637793277094
    - type: nauc_ndcg_at_1_diff1
      value: 91.52066384640422
    - type: nauc_ndcg_at_1_max
      value: 57.639765701269155
    - type: nauc_ndcg_at_1_std
      value: 1.4959323136999483
    - type: nauc_ndcg_at_20_diff1
      value: 90.55301735430407
    - type: nauc_ndcg_at_20_max
      value: 65.65827895142017
    - type: nauc_ndcg_at_20_std
      value: 9.990699968426656
    - type: nauc_ndcg_at_3_diff1
      value: 89.96356996211253
    - type: nauc_ndcg_at_3_max
      value: 64.03584890384899
    - type: nauc_ndcg_at_3_std
      value: 4.295929350921648
    - type: nauc_ndcg_at_5_diff1
      value: 90.89636835182858
    - type: nauc_ndcg_at_5_max
      value: 65.69109133325729
    - type: nauc_ndcg_at_5_std
      value: 7.251888023539761
    - type: nauc_precision_at_1000_diff1
      value: .nan
    - type: nauc_precision_at_1000_max
      value: .nan
    - type: nauc_precision_at_1000_std
      value: .nan
    - type: nauc_precision_at_100_diff1
      value: 89.89595838335227
    - type: nauc_precision_at_100_max
      value: 98.13258636787991
    - type: nauc_precision_at_100_std
      value: 95.88168600773692
    - type: nauc_precision_at_10_diff1
      value: 91.80313150901407
    - type: nauc_precision_at_10_max
      value: 84.73120017237689
    - type: nauc_precision_at_10_std
      value: 30.963154492565963
    - type: nauc_precision_at_1_diff1
      value: 91.52066384640422
    - type: nauc_precision_at_1_max
      value: 57.639765701269155
    - type: nauc_precision_at_1_std
      value: 1.4959323136999483
    - type: nauc_precision_at_20_diff1
      value: 88.51379632312695
    - type: nauc_precision_at_20_max
      value: 93.89870890885065
    - type: nauc_precision_at_20_std
      value: 66.06458675424169
    - type: nauc_precision_at_3_diff1
      value: 88.5919911035577
    - type: nauc_precision_at_3_max
      value: 70.47755431760744
    - type: nauc_precision_at_3_std
      value: 7.324877516549845
    - type: nauc_precision_at_5_diff1
      value: 91.68000533546777
    - type: nauc_precision_at_5_max
      value: 78.08323329331722
    - type: nauc_precision_at_5_std
      value: 18.784847272242132
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: 89.8959583833538
    - type: nauc_recall_at_100_max
      value: 98.13258636788038
    - type: nauc_recall_at_100_std
      value: 95.88168600773625
    - type: nauc_recall_at_10_diff1
      value: 91.80313150901372
    - type: nauc_recall_at_10_max
      value: 84.73120017237645
    - type: nauc_recall_at_10_std
      value: 30.963154492566204
    - type: nauc_recall_at_1_diff1
      value: 91.52066384640422
    - type: nauc_recall_at_1_max
      value: 57.639765701269155
    - type: nauc_recall_at_1_std
      value: 1.4959323136999483
    - type: nauc_recall_at_20_diff1
      value: 88.51379632312701
    - type: nauc_recall_at_20_max
      value: 93.89870890885045
    - type: nauc_recall_at_20_std
      value: 66.0645867542416
    - type: nauc_recall_at_3_diff1
      value: 88.59199110355753
    - type: nauc_recall_at_3_max
      value: 70.47755431760724
    - type: nauc_recall_at_3_std
      value: 7.324877516549637
    - type: nauc_recall_at_5_diff1
      value: 91.68000533546758
    - type: nauc_recall_at_5_max
      value: 78.08323329331743
    - type: nauc_recall_at_5_std
      value: 18.784847272242207
    - type: ndcg_at_1
      value: 85.0
    - type: ndcg_at_10
      value: 89.97800000000001
    - type: ndcg_at_100
      value: 90.842
    - type: ndcg_at_1000
      value: 91.00399999999999
    - type: ndcg_at_20
      value: 90.561
    - type: ndcg_at_3
      value: 88.626
    - type: ndcg_at_5
      value: 89.401
    - type: precision_at_1
      value: 85.0
    - type: precision_at_10
      value: 9.48
    - type: precision_at_100
      value: 0.9860000000000001
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.855
    - type: precision_at_3
      value: 30.367
    - type: precision_at_5
      value: 18.6
    - type: recall_at_1
      value: 85.0
    - type: recall_at_10
      value: 94.8
    - type: recall_at_100
      value: 98.6
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 97.1
    - type: recall_at_3
      value: 91.10000000000001
    - type: recall_at_5
      value: 93.0
    task:
      type: Retrieval
  - dataset:
      config: go
      name: MTEB CodeSearchNetRetrieval (go)
      revision: fdc6a9e39575768c27eb8a2a5f702bf846eb4759
      split: test
      type: code-search-net/code_search_net
    metrics:
    - type: main_score
      value: 97.505
    - type: map_at_1
      value: 94.39999999999999
    - type: map_at_10
      value: 96.808
    - type: map_at_100
      value: 96.82300000000001
    - type: map_at_1000
      value: 96.824
    - type: map_at_20
      value: 96.813
    - type: map_at_3
      value: 96.733
    - type: map_at_5
      value: 96.808
    - type: mrr_at_1
      value: 94.39999999999999
    - type: mrr_at_10
      value: 96.80833333333334
    - type: mrr_at_100
      value: 96.82307275541795
    - type: mrr_at_1000
      value: 96.82390608875127
    - type: mrr_at_20
      value: 96.81333333333333
    - type: mrr_at_3
      value: 96.73333333333333
    - type: mrr_at_5
      value: 96.80833333333334
    - type: nauc_map_at_1000_diff1
      value: 97.27333921447281
    - type: nauc_map_at_1000_max
      value: 40.53177582941515
    - type: nauc_map_at_1000_std
      value: -9.68497853001627
    - type: nauc_map_at_100_diff1
      value: 97.27405443932574
    - type: nauc_map_at_100_max
      value: 40.54737481852009
    - type: nauc_map_at_100_std
      value: -9.663493618536894
    - type: nauc_map_at_10_diff1
      value: 97.28664311677696
    - type: nauc_map_at_10_max
      value: 40.81115474910615
    - type: nauc_map_at_10_std
      value: -10.076963770712538
    - type: nauc_map_at_1_diff1
      value: 96.90709617180208
    - type: nauc_map_at_1_max
      value: 33.33833533413366
    - type: nauc_map_at_1_std
      value: -6.367547018807825
    - type: nauc_map_at_20_diff1
      value: 97.28238575765059
    - type: nauc_map_at_20_max
      value: 40.71828522203857
    - type: nauc_map_at_20_std
      value: -9.912064546879208
    - type: nauc_map_at_3_diff1
      value: 97.34893957583056
    - type: nauc_map_at_3_max
      value: 41.67881438289635
    - type: nauc_map_at_3_std
      value: -9.509041711922595
    - type: nauc_map_at_5_diff1
      value: 97.28664311677696
    - type: nauc_map_at_5_max
      value: 40.81115474910615
    - type: nauc_map_at_5_std
      value: -10.076963770712538
    - type: nauc_mrr_at_1000_diff1
      value: 97.27333921447281
    - type: nauc_mrr_at_1000_max
      value: 40.53177582941515
    - type: nauc_mrr_at_1000_std
      value: -9.68497853001627
    - type: nauc_mrr_at_100_diff1
      value: 97.27405443932574
    - type: nauc_mrr_at_100_max
      value: 40.54737481852009
    - type: nauc_mrr_at_100_std
      value: -9.663493618536894
    - type: nauc_mrr_at_10_diff1
      value: 97.28664311677696
    - type: nauc_mrr_at_10_max
      value: 40.81115474910615
    - type: nauc_mrr_at_10_std
      value: -10.076963770712538
    - type: nauc_mrr_at_1_diff1
      value: 96.90709617180208
    - type: nauc_mrr_at_1_max
      value: 33.33833533413366
    - type: nauc_mrr_at_1_std
      value: -6.367547018807825
    - type: nauc_mrr_at_20_diff1
      value: 97.28238575765059
    - type: nauc_mrr_at_20_max
      value: 40.71828522203857
    - type: nauc_mrr_at_20_std
      value: -9.912064546879208
    - type: nauc_mrr_at_3_diff1
      value: 97.34893957583056
    - type: nauc_mrr_at_3_max
      value: 41.67881438289635
    - type: nauc_mrr_at_3_std
      value: -9.509041711922595
    - type: nauc_mrr_at_5_diff1
      value: 97.28664311677696
    - type: nauc_mrr_at_5_max
      value: 40.81115474910615
    - type: nauc_mrr_at_5_std
      value: -10.076963770712538
    - type: nauc_ndcg_at_1000_diff1
      value: 97.33376561125955
    - type: nauc_ndcg_at_1000_max
      value: 41.75737985174807
    - type: nauc_ndcg_at_1000_std
      value: -10.511570027083764
    - type: nauc_ndcg_at_100_diff1
      value: 97.34974235237087
    - type: nauc_ndcg_at_100_max
      value: 42.106384150737576
    - type: nauc_ndcg_at_100_std
      value: -10.015808519562162
    - type: nauc_ndcg_at_10_diff1
      value: 97.43761675418024
    - type: nauc_ndcg_at_10_max
      value: 43.92682037582593
    - type: nauc_ndcg_at_10_std
      value: -13.11593389665566
    - type: nauc_ndcg_at_1_diff1
      value: 96.90709617180208
    - type: nauc_ndcg_at_1_max
      value: 33.33833533413366
    - type: nauc_ndcg_at_1_std
      value: -6.367547018807825
    - type: nauc_ndcg_at_20_diff1
      value: 97.41401672160926
    - type: nauc_ndcg_at_20_max
      value: 43.410375824598866
    - type: nauc_ndcg_at_20_std
      value: -12.175971519065602
    - type: nauc_ndcg_at_3_diff1
      value: 97.56379014982639
    - type: nauc_ndcg_at_3_max
      value: 45.63427566353211
    - type: nauc_ndcg_at_3_std
      value: -11.748268623706998
    - type: nauc_ndcg_at_5_diff1
      value: 97.43761675418024
    - type: nauc_ndcg_at_5_max
      value: 43.92682037582593
    - type: nauc_ndcg_at_5_std
      value: -13.11593389665566
    - type: nauc_precision_at_1000_diff1
      value: .nan
    - type: nauc_precision_at_1000_max
      value: .nan
    - type: nauc_precision_at_1000_std
      value: .nan
    - type: nauc_precision_at_100_diff1
      value: 100.0
    - type: nauc_precision_at_100_max
      value: 100.0
    - type: nauc_precision_at_100_std
      value: 72.22222222222335
    - type: nauc_precision_at_10_diff1
      value: 100.0
    - type: nauc_precision_at_10_max
      value: 97.38562091503177
    - type: nauc_precision_at_10_std
      value: -69.85060690943635
    - type: nauc_precision_at_1_diff1
      value: 96.90709617180208
    - type: nauc_precision_at_1_max
      value: 33.33833533413366
    - type: nauc_precision_at_1_std
      value: -6.367547018807825
    - type: nauc_precision_at_20_diff1
      value: 100.0
    - type: nauc_precision_at_20_max
      value: 96.73202614378965
    - type: nauc_precision_at_20_std
      value: -58.52007469655789
    - type: nauc_precision_at_3_diff1
      value: 100.0
    - type: nauc_precision_at_3_max
      value: 90.34197012138252
    - type: nauc_precision_at_3_std
      value: -38.159430438841106
    - type: nauc_precision_at_5_diff1
      value: 100.0
    - type: nauc_precision_at_5_max
      value: 97.38562091503177
    - type: nauc_precision_at_5_std
      value: -69.85060690943635
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: 100.0
    - type: nauc_recall_at_100_max
      value: 100.0
    - type: nauc_recall_at_100_std
      value: 72.22222222222368
    - type: nauc_recall_at_10_diff1
      value: 100.0
    - type: nauc_recall_at_10_max
      value: 97.38562091503128
    - type: nauc_recall_at_10_std
      value: -69.85060690942723
    - type: nauc_recall_at_1_diff1
      value: 96.90709617180208
    - type: nauc_recall_at_1_max
      value: 33.33833533413366
    - type: nauc_recall_at_1_std
      value: -6.367547018807825
    - type: nauc_recall_at_20_diff1
      value: 100.0
    - type: nauc_recall_at_20_max
      value: 96.7320261437891
    - type: nauc_recall_at_20_std
      value: -58.52007469654495
    - type: nauc_recall_at_3_diff1
      value: 100.0
    - type: nauc_recall_at_3_max
      value: 90.34197012138186
    - type: nauc_recall_at_3_std
      value: -38.15943043884293
    - type: nauc_recall_at_5_diff1
      value: 100.0
    - type: nauc_recall_at_5_max
      value: 97.38562091503128
    - type: nauc_recall_at_5_std
      value: -69.85060690942723
    - type: ndcg_at_1
      value: 94.39999999999999
    - type: ndcg_at_10
      value: 97.505
    - type: ndcg_at_100
      value: 97.588
    - type: ndcg_at_1000
      value: 97.602
    - type: ndcg_at_20
      value: 97.528
    - type: ndcg_at_3
      value: 97.37599999999999
    - type: ndcg_at_5
      value: 97.505
    - type: precision_at_1
      value: 94.39999999999999
    - type: precision_at_10
      value: 9.950000000000001
    - type: precision_at_100
      value: 0.9990000000000001
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.9799999999999995
    - type: precision_at_3
      value: 33.067
    - type: precision_at_5
      value: 19.900000000000002
    - type: recall_at_1
      value: 94.39999999999999
    - type: recall_at_10
      value: 99.5
    - type: recall_at_100
      value: 99.9
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 99.6
    - type: recall_at_3
      value: 99.2
    - type: recall_at_5
      value: 99.5
    task:
      type: Retrieval
  - dataset:
      config: ruby
      name: MTEB CodeSearchNetRetrieval (ruby)
      revision: fdc6a9e39575768c27eb8a2a5f702bf846eb4759
      split: test
      type: code-search-net/code_search_net
    metrics:
    - type: main_score
      value: 92.83200000000001
    - type: map_at_1
      value: 87.7
    - type: map_at_10
      value: 91.496
    - type: map_at_100
      value: 91.582
    - type: map_at_1000
      value: 91.58699999999999
    - type: map_at_20
      value: 91.55
    - type: map_at_3
      value: 91.167
    - type: map_at_5
      value: 91.39699999999999
    - type: mrr_at_1
      value: 87.7
    - type: mrr_at_10
      value: 91.49646825396826
    - type: mrr_at_100
      value: 91.58154350757961
    - type: mrr_at_1000
      value: 91.58711452954685
    - type: mrr_at_20
      value: 91.54990637415563
    - type: mrr_at_3
      value: 91.16666666666669
    - type: mrr_at_5
      value: 91.39666666666668
    - type: nauc_map_at_1000_diff1
      value: 90.01079432013319
    - type: nauc_map_at_1000_max
      value: 76.25110199586136
    - type: nauc_map_at_1000_std
      value: 21.95406993804367
    - type: nauc_map_at_100_diff1
      value: 90.01415529036015
    - type: nauc_map_at_100_max
      value: 76.26710387746687
    - type: nauc_map_at_100_std
      value: 21.991489847497782
    - type: nauc_map_at_10_diff1
      value: 89.88101575524061
    - type: nauc_map_at_10_max
      value: 76.40985435650425
    - type: nauc_map_at_10_std
      value: 22.554999001780367
    - type: nauc_map_at_1_diff1
      value: 90.45505123995243
    - type: nauc_map_at_1_max
      value: 70.27343537229697
    - type: nauc_map_at_1_std
      value: 10.378716550076664
    - type: nauc_map_at_20_diff1
      value: 90.03751820546869
    - type: nauc_map_at_20_max
      value: 76.35756916603282
    - type: nauc_map_at_20_std
      value: 22.15447723869247
    - type: nauc_map_at_3_diff1
      value: 89.90841801530556
    - type: nauc_map_at_3_max
      value: 76.90028137713493
    - type: nauc_map_at_3_std
      value: 21.843737118482988
    - type: nauc_map_at_5_diff1
      value: 89.93455699311205
    - type: nauc_map_at_5_max
      value: 76.52511956217161
    - type: nauc_map_at_5_std
      value: 22.235121325826523
    - type: nauc_mrr_at_1000_diff1
      value: 90.01079432013319
    - type: nauc_mrr_at_1000_max
      value: 76.25110199586136
    - type: nauc_mrr_at_1000_std
      value: 21.95406993804367
    - type: nauc_mrr_at_100_diff1
      value: 90.01415529036015
    - type: nauc_mrr_at_100_max
      value: 76.26710387746687
    - type: nauc_mrr_at_100_std
      value: 21.991489847497782
    - type: nauc_mrr_at_10_diff1
      value: 89.88101575524061
    - type: nauc_mrr_at_10_max
      value: 76.40985435650425
    - type: nauc_mrr_at_10_std
      value: 22.554999001780367
    - type: nauc_mrr_at_1_diff1
      value: 90.45505123995243
    - type: nauc_mrr_at_1_max
      value: 70.27343537229697
    - type: nauc_mrr_at_1_std
      value: 10.378716550076664
    - type: nauc_mrr_at_20_diff1
      value: 90.03751820546869
    - type: nauc_mrr_at_20_max
      value: 76.35756916603282
    - type: nauc_mrr_at_20_std
      value: 22.15447723869247
    - type: nauc_mrr_at_3_diff1
      value: 89.90841801530556
    - type: nauc_mrr_at_3_max
      value: 76.90028137713493
    - type: nauc_mrr_at_3_std
      value: 21.843737118482988
    - type: nauc_mrr_at_5_diff1
      value: 89.93455699311205
    - type: nauc_mrr_at_5_max
      value: 76.52511956217161
    - type: nauc_mrr_at_5_std
      value: 22.235121325826523
    - type: nauc_ndcg_at_1000_diff1
      value: 90.0388376789719
    - type: nauc_ndcg_at_1000_max
      value: 77.4286773752732
    - type: nauc_ndcg_at_1000_std
      value: 24.905596869880494
    - type: nauc_ndcg_at_100_diff1
      value: 90.1570526184995
    - type: nauc_ndcg_at_100_max
      value: 77.97459201928656
    - type: nauc_ndcg_at_100_std
      value: 26.36432448776628
    - type: nauc_ndcg_at_10_diff1
      value: 89.60631662977974
    - type: nauc_ndcg_at_10_max
      value: 78.82415924492551
    - type: nauc_ndcg_at_10_std
      value: 29.259897960486477
    - type: nauc_ndcg_at_1_diff1
      value: 90.45505123995243
    - type: nauc_ndcg_at_1_max
      value: 70.27343537229697
    - type: nauc_ndcg_at_1_std
      value: 10.378716550076664
    - type: nauc_ndcg_at_20_diff1
      value: 90.34712330725388
    - type: nauc_ndcg_at_20_max
      value: 78.65263241067265
    - type: nauc_ndcg_at_20_std
      value: 27.634547611895844
    - type: nauc_ndcg_at_3_diff1
      value: 89.69525439370882
    - type: nauc_ndcg_at_3_max
      value: 79.68227200895333
    - type: nauc_ndcg_at_3_std
      value: 27.16978363425553
    - type: nauc_ndcg_at_5_diff1
      value: 89.74542043203509
    - type: nauc_ndcg_at_5_max
      value: 79.0789328591088
    - type: nauc_ndcg_at_5_std
      value: 28.35584817342069
    - type: nauc_precision_at_1000_diff1
      value: .nan
    - type: nauc_precision_at_1000_max
      value: .nan
    - type: nauc_precision_at_1000_std
      value: .nan
    - type: nauc_precision_at_100_diff1
      value: 94.84665661136168
    - type: nauc_precision_at_100_max
      value: 100.0
    - type: nauc_precision_at_100_std
      value: 86.96760755584259
    - type: nauc_precision_at_10_diff1
      value: 87.527719421102
    - type: nauc_precision_at_10_max
      value: 96.1922268907559
    - type: nauc_precision_at_10_std
      value: 80.32942343604061
    - type: nauc_precision_at_1_diff1
      value: 90.45505123995243
    - type: nauc_precision_at_1_max
      value: 70.27343537229697
    - type: nauc_precision_at_1_std
      value: 10.378716550076664
    - type: nauc_precision_at_20_diff1
      value: 95.5746187363837
    - type: nauc_precision_at_20_max
      value: 100.0
    - type: nauc_precision_at_20_std
      value: 78.69397759103582
    - type: nauc_precision_at_3_diff1
      value: 88.63450141961569
    - type: nauc_precision_at_3_max
      value: 93.64984088873672
    - type: nauc_precision_at_3_std
      value: 54.208350006669704
    - type: nauc_precision_at_5_diff1
      value: 88.62314156431772
    - type: nauc_precision_at_5_max
      value: 94.5892887069356
    - type: nauc_precision_at_5_std
      value: 66.80552562905466
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: 94.8466566113629
    - type: nauc_recall_at_100_max
      value: 100.0
    - type: nauc_recall_at_100_std
      value: 86.96760755584259
    - type: nauc_recall_at_10_diff1
      value: 87.52771942110216
    - type: nauc_recall_at_10_max
      value: 96.19222689075643
    - type: nauc_recall_at_10_std
      value: 80.32942343604144
    - type: nauc_recall_at_1_diff1
      value: 90.45505123995243
    - type: nauc_recall_at_1_max
      value: 70.27343537229697
    - type: nauc_recall_at_1_std
      value: 10.378716550076664
    - type: nauc_recall_at_20_diff1
      value: 95.57461873638331
    - type: nauc_recall_at_20_max
      value: 100.0
    - type: nauc_recall_at_20_std
      value: 78.69397759103617
    - type: nauc_recall_at_3_diff1
      value: 88.63450141961552
    - type: nauc_recall_at_3_max
      value: 93.64984088873658
    - type: nauc_recall_at_3_std
      value: 54.20835000666945
    - type: nauc_recall_at_5_diff1
      value: 88.62314156431815
    - type: nauc_recall_at_5_max
      value: 94.58928870693578
    - type: nauc_recall_at_5_std
      value: 66.80552562905518
    - type: ndcg_at_1
      value: 87.7
    - type: ndcg_at_10
      value: 92.83200000000001
    - type: ndcg_at_100
      value: 93.242
    - type: ndcg_at_1000
      value: 93.403
    - type: ndcg_at_20
      value: 93.03099999999999
    - type: ndcg_at_3
      value: 92.186
    - type: ndcg_at_5
      value: 92.599
    - type: precision_at_1
      value: 87.7
    - type: precision_at_10
      value: 9.68
    - type: precision_at_100
      value: 0.987
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.88
    - type: precision_at_3
      value: 31.7
    - type: precision_at_5
      value: 19.220000000000002
    - type: recall_at_1
      value: 87.7
    - type: recall_at_10
      value: 96.8
    - type: recall_at_100
      value: 98.7
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 97.6
    - type: recall_at_3
      value: 95.1
    - type: recall_at_5
      value: 96.1
    task:
      type: Retrieval
  - dataset:
      config: java
      name: MTEB CodeSearchNetRetrieval (java)
      revision: fdc6a9e39575768c27eb8a2a5f702bf846eb4759
      split: test
      type: code-search-net/code_search_net
    metrics:
    - type: main_score
      value: 94.50699999999999
    - type: map_at_1
      value: 91.4
    - type: map_at_10
      value: 93.67699999999999
    - type: map_at_100
      value: 93.746
    - type: map_at_1000
      value: 93.753
    - type: map_at_20
      value: 93.706
    - type: map_at_3
      value: 93.43299999999999
    - type: map_at_5
      value: 93.593
    - type: mrr_at_1
      value: 91.4
    - type: mrr_at_10
      value: 93.67706349206348
    - type: mrr_at_100
      value: 93.74605961033865
    - type: mrr_at_1000
      value: 93.75307690063596
    - type: mrr_at_20
      value: 93.70575802250723
    - type: mrr_at_3
      value: 93.43333333333334
    - type: mrr_at_5
      value: 93.59333333333333
    - type: nauc_map_at_1000_diff1
      value: 91.96988867062096
    - type: nauc_map_at_1000_max
      value: 80.8753031650805
    - type: nauc_map_at_1000_std
      value: 54.38722696006999
    - type: nauc_map_at_100_diff1
      value: 91.97889892956285
    - type: nauc_map_at_100_max
      value: 80.89676220389073
    - type: nauc_map_at_100_std
      value: 54.4132005408842
    - type: nauc_map_at_10_diff1
      value: 92.00847920033857
    - type: nauc_map_at_10_max
      value: 81.10521744502199
    - type: nauc_map_at_10_std
      value: 54.68790929661031
    - type: nauc_map_at_1_diff1
      value: 89.60328317373455
    - type: nauc_map_at_1_max
      value: 72.47899159663876
    - type: nauc_map_at_1_std
      value: 44.26584587323289
    - type: nauc_map_at_20_diff1
      value: 92.02464449562623
    - type: nauc_map_at_20_max
      value: 81.01907888930855
    - type: nauc_map_at_20_std
      value: 54.571031073682384
    - type: nauc_map_at_3_diff1
      value: 92.09358870451744
    - type: nauc_map_at_3_max
      value: 81.7007682937812
    - type: nauc_map_at_3_std
      value: 55.27046690080435
    - type: nauc_map_at_5_diff1
      value: 92.11292217199042
    - type: nauc_map_at_5_max
      value: 81.3521575817287
    - type: nauc_map_at_5_std
      value: 54.99813695856418
    - type: nauc_mrr_at_1000_diff1
      value: 91.96988867062096
    - type: nauc_mrr_at_1000_max
      value: 80.8753031650805
    - type: nauc_mrr_at_1000_std
      value: 54.38722696006999
    - type: nauc_mrr_at_100_diff1
      value: 91.97889892956285
    - type: nauc_mrr_at_100_max
      value: 80.89676220389073
    - type: nauc_mrr_at_100_std
      value: 54.4132005408842
    - type: nauc_mrr_at_10_diff1
      value: 92.00847920033857
    - type: nauc_mrr_at_10_max
      value: 81.10521744502199
    - type: nauc_mrr_at_10_std
      value: 54.68790929661031
    - type: nauc_mrr_at_1_diff1
      value: 89.60328317373455
    - type: nauc_mrr_at_1_max
      value: 72.47899159663876
    - type: nauc_mrr_at_1_std
      value: 44.26584587323289
    - type: nauc_mrr_at_20_diff1
      value: 92.02464449562623
    - type: nauc_mrr_at_20_max
      value: 81.01907888930855
    - type: nauc_mrr_at_20_std
      value: 54.571031073682384
    - type: nauc_mrr_at_3_diff1
      value: 92.09358870451744
    - type: nauc_mrr_at_3_max
      value: 81.7007682937812
    - type: nauc_mrr_at_3_std
      value: 55.27046690080435
    - type: nauc_mrr_at_5_diff1
      value: 92.11292217199042
    - type: nauc_mrr_at_5_max
      value: 81.3521575817287
    - type: nauc_mrr_at_5_std
      value: 54.99813695856418
    - type: nauc_ndcg_at_1000_diff1
      value: 92.39514411202762
    - type: nauc_ndcg_at_1000_max
      value: 82.1789464930351
    - type: nauc_ndcg_at_1000_std
      value: 55.98331155582365
    - type: nauc_ndcg_at_100_diff1
      value: 92.60315730234768
    - type: nauc_ndcg_at_100_max
      value: 82.66640006867893
    - type: nauc_ndcg_at_100_std
      value: 56.60969071497668
    - type: nauc_ndcg_at_10_diff1
      value: 92.84846723213055
    - type: nauc_ndcg_at_10_max
      value: 83.93273402590029
    - type: nauc_ndcg_at_10_std
      value: 58.180048945638184
    - type: nauc_ndcg_at_1_diff1
      value: 89.60328317373455
    - type: nauc_ndcg_at_1_max
      value: 72.47899159663876
    - type: nauc_ndcg_at_1_std
      value: 44.26584587323289
    - type: nauc_ndcg_at_20_diff1
      value: 92.9455259704201
    - type: nauc_ndcg_at_20_max
      value: 83.57732205562041
    - type: nauc_ndcg_at_20_std
      value: 57.695601177913225
    - type: nauc_ndcg_at_3_diff1
      value: 93.02972756100402
    - type: nauc_ndcg_at_3_max
      value: 85.03899149427752
    - type: nauc_ndcg_at_3_std
      value: 59.29245146091644
    - type: nauc_ndcg_at_5_diff1
      value: 93.09702309051526
    - type: nauc_ndcg_at_5_max
      value: 84.49116159880194
    - type: nauc_ndcg_at_5_std
      value: 58.887701004976
    - type: nauc_precision_at_1000_diff1
      value: .nan
    - type: nauc_precision_at_1000_max
      value: .nan
    - type: nauc_precision_at_1000_std
      value: .nan
    - type: nauc_precision_at_100_diff1
      value: 100.0
    - type: nauc_precision_at_100_max
      value: 100.0
    - type: nauc_precision_at_100_std
      value: 79.24836601307065
    - type: nauc_precision_at_10_diff1
      value: 97.6423902894492
    - type: nauc_precision_at_10_max
      value: 100.0
    - type: nauc_precision_at_10_std
      value: 78.07345160286333
    - type: nauc_precision_at_1_diff1
      value: 89.60328317373455
    - type: nauc_precision_at_1_max
      value: 72.47899159663876
    - type: nauc_precision_at_1_std
      value: 44.26584587323289
    - type: nauc_precision_at_20_diff1
      value: 99.47712418300702
    - type: nauc_precision_at_20_max
      value: 100.0
    - type: nauc_precision_at_20_std
      value: 77.69374416433266
    - type: nauc_precision_at_3_diff1
      value: 97.06316635181199
    - type: nauc_precision_at_3_max
      value: 99.35400516795852
    - type: nauc_precision_at_3_std
      value: 76.56070180010008
    - type: nauc_precision_at_5_diff1
      value: 98.03532524120767
    - type: nauc_precision_at_5_max
      value: 100.0
    - type: nauc_precision_at_5_std
      value: 78.16163502438044
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: 100.0
    - type: nauc_recall_at_100_max
      value: 100.0
    - type: nauc_recall_at_100_std
      value: 79.24836601307193
    - type: nauc_recall_at_10_diff1
      value: 97.64239028944921
    - type: nauc_recall_at_10_max
      value: 100.0
    - type: nauc_recall_at_10_std
      value: 78.07345160286356
    - type: nauc_recall_at_1_diff1
      value: 89.60328317373455
    - type: nauc_recall_at_1_max
      value: 72.47899159663876
    - type: nauc_recall_at_1_std
      value: 44.26584587323289
    - type: nauc_recall_at_20_diff1
      value: 99.47712418300625
    - type: nauc_recall_at_20_max
      value: 100.0
    - type: nauc_recall_at_20_std
      value: 77.69374416433251
    - type: nauc_recall_at_3_diff1
      value: 97.06316635181186
    - type: nauc_recall_at_3_max
      value: 99.35400516795868
    - type: nauc_recall_at_3_std
      value: 76.56070180009995
    - type: nauc_recall_at_5_diff1
      value: 98.03532524120769
    - type: nauc_recall_at_5_max
      value: 100.0
    - type: nauc_recall_at_5_std
      value: 78.16163502438029
    - type: ndcg_at_1
      value: 91.4
    - type: ndcg_at_10
      value: 94.50699999999999
    - type: ndcg_at_100
      value: 94.90899999999999
    - type: ndcg_at_1000
      value: 95.048
    - type: ndcg_at_20
      value: 94.626
    - type: ndcg_at_3
      value: 94.021
    - type: ndcg_at_5
      value: 94.31
    - type: precision_at_1
      value: 91.4
    - type: precision_at_10
      value: 9.700000000000001
    - type: precision_at_100
      value: 0.9900000000000001
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.875
    - type: precision_at_3
      value: 31.900000000000002
    - type: precision_at_5
      value: 19.28
    - type: recall_at_1
      value: 91.4
    - type: recall_at_10
      value: 97.0
    - type: recall_at_100
      value: 99.0
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 97.5
    - type: recall_at_3
      value: 95.7
    - type: recall_at_5
      value: 96.39999999999999
    task:
      type: Retrieval
  - dataset:
      config: php
      name: MTEB CodeSearchNetRetrieval (php)
      revision: fdc6a9e39575768c27eb8a2a5f702bf846eb4759
      split: test
      type: code-search-net/code_search_net
    metrics:
    - type: main_score
      value: 92.288
    - type: map_at_1
      value: 85.5
    - type: map_at_10
      value: 90.312
    - type: map_at_100
      value: 90.351
    - type: map_at_1000
      value: 90.35300000000001
    - type: map_at_20
      value: 90.335
    - type: map_at_3
      value: 89.583
    - type: map_at_5
      value: 90.098
    - type: mrr_at_1
      value: 85.5
    - type: mrr_at_10
      value: 90.31230158730165
    - type: mrr_at_100
      value: 90.35130514931716
    - type: mrr_at_1000
      value: 90.3533725076738
    - type: mrr_at_20
      value: 90.33520202020209
    - type: mrr_at_3
      value: 89.58333333333339
    - type: mrr_at_5
      value: 90.09833333333341
    - type: nauc_map_at_1000_diff1
      value: 93.85544381917798
    - type: nauc_map_at_1000_max
      value: 40.90472409247008
    - type: nauc_map_at_1000_std
      value: -16.463521873713017
    - type: nauc_map_at_100_diff1
      value: 93.85516451221955
    - type: nauc_map_at_100_max
      value: 40.917784128196594
    - type: nauc_map_at_100_std
      value: -16.437783475285407
    - type: nauc_map_at_10_diff1
      value: 93.821658302016
    - type: nauc_map_at_10_max
      value: 41.1631023862988
    - type: nauc_map_at_10_std
      value: -15.954316262724616
    - type: nauc_map_at_1_diff1
      value: 94.51659451659448
    - type: nauc_map_at_1_max
      value: 35.95691130574863
    - type: nauc_map_at_1_std
      value: -25.156884459210044
    - type: nauc_map_at_20_diff1
      value: 93.83850503661382
    - type: nauc_map_at_20_max
      value: 41.019314488512634
    - type: nauc_map_at_20_std
      value: -16.237689928755426
    - type: nauc_map_at_3_diff1
      value: 93.80837080594951
    - type: nauc_map_at_3_max
      value: 40.635458703255324
    - type: nauc_map_at_3_std
      value: -17.033014335677798
    - type: nauc_map_at_5_diff1
      value: 93.9020242116256
    - type: nauc_map_at_5_max
      value: 41.174017150832654
    - type: nauc_map_at_5_std
      value: -16.30029652673213
    - type: nauc_mrr_at_1000_diff1
      value: 93.85544381917798
    - type: nauc_mrr_at_1000_max
      value: 40.90472409247008
    - type: nauc_mrr_at_1000_std
      value: -16.463521873713017
    - type: nauc_mrr_at_100_diff1
      value: 93.85516451221955
    - type: nauc_mrr_at_100_max
      value: 40.917784128196594
    - type: nauc_mrr_at_100_std
      value: -16.437783475285407
    - type: nauc_mrr_at_10_diff1
      value: 93.821658302016
    - type: nauc_mrr_at_10_max
      value: 41.1631023862988
    - type: nauc_mrr_at_10_std
      value: -15.954316262724616
    - type: nauc_mrr_at_1_diff1
      value: 94.51659451659448
    - type: nauc_mrr_at_1_max
      value: 35.95691130574863
    - type: nauc_mrr_at_1_std
      value: -25.156884459210044
    - type: nauc_mrr_at_20_diff1
      value: 93.83850503661382
    - type: nauc_mrr_at_20_max
      value: 41.019314488512634
    - type: nauc_mrr_at_20_std
      value: -16.237689928755426
    - type: nauc_mrr_at_3_diff1
      value: 93.80837080594951
    - type: nauc_mrr_at_3_max
      value: 40.635458703255324
    - type: nauc_mrr_at_3_std
      value: -17.033014335677798
    - type: nauc_mrr_at_5_diff1
      value: 93.9020242116256
    - type: nauc_mrr_at_5_max
      value: 41.174017150832654
    - type: nauc_mrr_at_5_std
      value: -16.30029652673213
    - type: nauc_ndcg_at_1000_diff1
      value: 93.77150134331873
    - type: nauc_ndcg_at_1000_max
      value: 42.34949722008596
    - type: nauc_ndcg_at_1000_std
      value: -13.688355172335143
    - type: nauc_ndcg_at_100_diff1
      value: 93.77562254293132
    - type: nauc_ndcg_at_100_max
      value: 42.992049705674965
    - type: nauc_ndcg_at_100_std
      value: -12.42122424240366
    - type: nauc_ndcg_at_10_diff1
      value: 93.54925638165832
    - type: nauc_ndcg_at_10_max
      value: 44.555550459089986
    - type: nauc_ndcg_at_10_std
      value: -9.337958348868717
    - type: nauc_ndcg_at_1_diff1
      value: 94.51659451659448
    - type: nauc_ndcg_at_1_max
      value: 35.95691130574863
    - type: nauc_ndcg_at_1_std
      value: -25.156884459210044
    - type: nauc_ndcg_at_20_diff1
      value: 93.61932694899781
    - type: nauc_ndcg_at_20_max
      value: 43.96879091200643
    - type: nauc_ndcg_at_20_std
      value: -10.495064090760449
    - type: nauc_ndcg_at_3_diff1
      value: 93.62155771108924
    - type: nauc_ndcg_at_3_max
      value: 42.893378116364104
    - type: nauc_ndcg_at_3_std
      value: -12.73029168651973
    - type: nauc_ndcg_at_5_diff1
      value: 93.78967591680556
    - type: nauc_ndcg_at_5_max
      value: 44.44155136825693
    - type: nauc_ndcg_at_5_std
      value: -10.579261977804965
    - type: nauc_precision_at_1000_diff1
      value: .nan
    - type: nauc_precision_at_1000_max
      value: .nan
    - type: nauc_precision_at_1000_std
      value: .nan
    - type: nauc_precision_at_100_diff1
      value: 94.39775910363963
    - type: nauc_precision_at_100_max
      value: 100.0
    - type: nauc_precision_at_100_std
      value: 100.0
    - type: nauc_precision_at_10_diff1
      value: 89.1387927720107
    - type: nauc_precision_at_10_max
      value: 100.0
    - type: nauc_precision_at_10_std
      value: 100.0
    - type: nauc_precision_at_1_diff1
      value: 94.51659451659448
    - type: nauc_precision_at_1_max
      value: 35.95691130574863
    - type: nauc_precision_at_1_std
      value: -25.156884459210044
    - type: nauc_precision_at_20_diff1
      value: 89.61251167133568
    - type: nauc_precision_at_20_max
      value: 100.0
    - type: nauc_precision_at_20_std
      value: 100.0
    - type: nauc_precision_at_3_diff1
      value: 92.71284271284276
    - type: nauc_precision_at_3_max
      value: 54.854426619132425
    - type: nauc_precision_at_3_std
      value: 10.262286732874971
    - type: nauc_precision_at_5_diff1
      value: 93.00449346405235
    - type: nauc_precision_at_5_max
      value: 73.52795284780564
    - type: nauc_precision_at_5_std
      value: 40.204831932772336
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: 94.39775910364197
    - type: nauc_recall_at_100_max
      value: 100.0
    - type: nauc_recall_at_100_std
      value: 100.0
    - type: nauc_recall_at_10_diff1
      value: 89.13879277201103
    - type: nauc_recall_at_10_max
      value: 100.0
    - type: nauc_recall_at_10_std
      value: 100.0
    - type: nauc_recall_at_1_diff1
      value: 94.51659451659448
    - type: nauc_recall_at_1_max
      value: 35.95691130574863
    - type: nauc_recall_at_1_std
      value: -25.156884459210044
    - type: nauc_recall_at_20_diff1
      value: 89.61251167133527
    - type: nauc_recall_at_20_max
      value: 100.0
    - type: nauc_recall_at_20_std
      value: 100.0
    - type: nauc_recall_at_3_diff1
      value: 92.71284271284289
    - type: nauc_recall_at_3_max
      value: 54.85442661913248
    - type: nauc_recall_at_3_std
      value: 10.262286732875078
    - type: nauc_recall_at_5_diff1
      value: 93.00449346405259
    - type: nauc_recall_at_5_max
      value: 73.52795284780608
    - type: nauc_recall_at_5_std
      value: 40.20483193277305
    - type: ndcg_at_1
      value: 85.5
    - type: ndcg_at_10
      value: 92.288
    - type: ndcg_at_100
      value: 92.49300000000001
    - type: ndcg_at_1000
      value: 92.574
    - type: ndcg_at_20
      value: 92.366
    - type: ndcg_at_3
      value: 90.851
    - type: ndcg_at_5
      value: 91.789
    - type: precision_at_1
      value: 85.5
    - type: precision_at_10
      value: 9.83
    - type: precision_at_100
      value: 0.993
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.93
    - type: precision_at_3
      value: 31.5
    - type: precision_at_5
      value: 19.36
    - type: recall_at_1
      value: 85.5
    - type: recall_at_10
      value: 98.3
    - type: recall_at_100
      value: 99.3
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 98.6
    - type: recall_at_3
      value: 94.5
    - type: recall_at_5
      value: 96.8
    task:
      type: Retrieval
  - dataset:
      config: default
      name: MTEB CodeTransOceanContest (default)
      revision: 20da4eb20a4b17300c0986ee148c90867a7f2a4d
      split: test
      type: CoIR-Retrieval/codetrans-contest
    metrics:
    - type: main_score
      value: 94.96
    - type: map_at_1
      value: 91.855
    - type: map_at_10
      value: 94.054
    - type: map_at_100
      value: 94.14099999999999
    - type: map_at_1000
      value: 94.145
    - type: map_at_20
      value: 94.124
    - type: map_at_3
      value: 93.741
    - type: map_at_5
      value: 93.92200000000001
    - type: mrr_at_1
      value: 91.8552036199095
    - type: mrr_at_10
      value: 94.05354449472097
    - type: mrr_at_100
      value: 94.14134449135241
    - type: mrr_at_1000
      value: 94.14490739440583
    - type: mrr_at_20
      value: 94.12357250592545
    - type: mrr_at_3
      value: 93.74057315233786
    - type: mrr_at_5
      value: 93.92156862745098
    - type: nauc_map_at_1000_diff1
      value: 92.55002145543314
    - type: nauc_map_at_1000_max
      value: 83.9544206201828
    - type: nauc_map_at_1000_std
      value: 13.818812515717433
    - type: nauc_map_at_100_diff1
      value: 92.55455211118941
    - type: nauc_map_at_100_max
      value: 83.96417863440325
    - type: nauc_map_at_100_std
      value: 13.871223041576354
    - type: nauc_map_at_10_diff1
      value: 92.58163562742963
    - type: nauc_map_at_10_max
      value: 84.20094910390932
    - type: nauc_map_at_10_std
      value: 14.827570038179012
    - type: nauc_map_at_1_diff1
      value: 90.8130775986731
    - type: nauc_map_at_1_max
      value: 81.11634573555125
    - type: nauc_map_at_1_std
      value: 6.18089047774425
    - type: nauc_map_at_20_diff1
      value: 92.57706925983298
    - type: nauc_map_at_20_max
      value: 84.01267551178432
    - type: nauc_map_at_20_std
      value: 13.990683573503478
    - type: nauc_map_at_3_diff1
      value: 92.76365755738576
    - type: nauc_map_at_3_max
      value: 84.18142235516605
    - type: nauc_map_at_3_std
      value: 11.473399520154405
    - type: nauc_map_at_5_diff1
      value: 92.74270433278942
    - type: nauc_map_at_5_max
      value: 84.12361744519418
    - type: nauc_map_at_5_std
      value: 14.668669421165939
    - type: nauc_mrr_at_1000_diff1
      value: 92.55002145543314
    - type: nauc_mrr_at_1000_max
      value: 83.9544206201828
    - type: nauc_mrr_at_1000_std
      value: 13.818812515717433
    - type: nauc_mrr_at_100_diff1
      value: 92.55455211118941
    - type: nauc_mrr_at_100_max
      value: 83.96417863440325
    - type: nauc_mrr_at_100_std
      value: 13.871223041576354
    - type: nauc_mrr_at_10_diff1
      value: 92.58163562742963
    - type: nauc_mrr_at_10_max
      value: 84.20094910390932
    - type: nauc_mrr_at_10_std
      value: 14.827570038179012
    - type: nauc_mrr_at_1_diff1
      value: 90.8130775986731
    - type: nauc_mrr_at_1_max
      value: 81.11634573555125
    - type: nauc_mrr_at_1_std
      value: 6.18089047774425
    - type: nauc_mrr_at_20_diff1
      value: 92.57706925983298
    - type: nauc_mrr_at_20_max
      value: 84.01267551178432
    - type: nauc_mrr_at_20_std
      value: 13.990683573503478
    - type: nauc_mrr_at_3_diff1
      value: 92.76365755738576
    - type: nauc_mrr_at_3_max
      value: 84.18142235516605
    - type: nauc_mrr_at_3_std
      value: 11.473399520154405
    - type: nauc_mrr_at_5_diff1
      value: 92.74270433278942
    - type: nauc_mrr_at_5_max
      value: 84.12361744519418
    - type: nauc_mrr_at_5_std
      value: 14.668669421165939
    - type: nauc_ndcg_at_1000_diff1
      value: 92.84906743720337
    - type: nauc_ndcg_at_1000_max
      value: 84.57965223804021
    - type: nauc_ndcg_at_1000_std
      value: 15.875602814166239
    - type: nauc_ndcg_at_100_diff1
      value: 92.94862546375188
    - type: nauc_ndcg_at_100_max
      value: 84.7943402354994
    - type: nauc_ndcg_at_100_std
      value: 17.046814945571057
    - type: nauc_ndcg_at_10_diff1
      value: 93.18693605775724
    - type: nauc_ndcg_at_10_max
      value: 85.99181169093023
    - type: nauc_ndcg_at_10_std
      value: 21.453785415429227
    - type: nauc_ndcg_at_1_diff1
      value: 90.8130775986731
    - type: nauc_ndcg_at_1_max
      value: 81.11634573555125
    - type: nauc_ndcg_at_1_std
      value: 6.18089047774425
    - type: nauc_ndcg_at_20_diff1
      value: 93.18182308386798
    - type: nauc_ndcg_at_20_max
      value: 85.29721008749047
    - type: nauc_ndcg_at_20_std
      value: 18.265825551265774
    - type: nauc_ndcg_at_3_diff1
      value: 93.5678392571751
    - type: nauc_ndcg_at_3_max
      value: 85.68193097329507
    - type: nauc_ndcg_at_3_std
      value: 13.252390138276315
    - type: nauc_ndcg_at_5_diff1
      value: 93.57442357672642
    - type: nauc_ndcg_at_5_max
      value: 85.65305420139138
    - type: nauc_ndcg_at_5_std
      value: 20.399096483631958
    - type: nauc_precision_at_1000_diff1
      value: 100.0
    - type: nauc_precision_at_1000_max
      value: 100.0
    - type: nauc_precision_at_1000_std
      value: 100.0
    - type: nauc_precision_at_100_diff1
      value: 100.0
    - type: nauc_precision_at_100_max
      value: 100.0
    - type: nauc_precision_at_100_std
      value: 100.0
    - type: nauc_precision_at_10_diff1
      value: 97.38693467336647
    - type: nauc_precision_at_10_max
      value: 100.0
    - type: nauc_precision_at_10_std
      value: 76.08263155229871
    - type: nauc_precision_at_1_diff1
      value: 90.8130775986731
    - type: nauc_precision_at_1_max
      value: 81.11634573555125
    - type: nauc_precision_at_1_std
      value: 6.18089047774425
    - type: nauc_precision_at_20_diff1
      value: 100.0
    - type: nauc_precision_at_20_max
      value: 100.0
    - type: nauc_precision_at_20_std
      value: 69.38924997712752
    - type: nauc_precision_at_3_diff1
      value: 97.09659408151863
    - type: nauc_precision_at_3_max
      value: 92.38060989189518
    - type: nauc_precision_at_3_std
      value: 20.8638963856334
    - type: nauc_precision_at_5_diff1
      value: 98.13352476669084
    - type: nauc_precision_at_5_max
      value: 94.1685830281352
    - type: nauc_precision_at_5_std
      value: 54.20627096881086
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: 100.0
    - type: nauc_recall_at_100_max
      value: 100.0
    - type: nauc_recall_at_100_std
      value: 100.0
    - type: nauc_recall_at_10_diff1
      value: 97.38693467336678
    - type: nauc_recall_at_10_max
      value: 100.0
    - type: nauc_recall_at_10_std
      value: 76.08263155229885
    - type: nauc_recall_at_1_diff1
      value: 90.8130775986731
    - type: nauc_recall_at_1_max
      value: 81.11634573555125
    - type: nauc_recall_at_1_std
      value: 6.18089047774425
    - type: nauc_recall_at_20_diff1
      value: 100.0
    - type: nauc_recall_at_20_max
      value: 100.0
    - type: nauc_recall_at_20_std
      value: 69.3892499771279
    - type: nauc_recall_at_3_diff1
      value: 97.09659408151863
    - type: nauc_recall_at_3_max
      value: 92.38060989189518
    - type: nauc_recall_at_3_std
      value: 20.863896385634074
    - type: nauc_recall_at_5_diff1
      value: 98.13352476669056
    - type: nauc_recall_at_5_max
      value: 94.16858302813482
    - type: nauc_recall_at_5_std
      value: 54.20627096881058
    - type: ndcg_at_1
      value: 91.855
    - type: ndcg_at_10
      value: 94.96
    - type: ndcg_at_100
      value: 95.357
    - type: ndcg_at_1000
      value: 95.422
    - type: ndcg_at_20
      value: 95.19800000000001
    - type: ndcg_at_3
      value: 94.306
    - type: ndcg_at_5
      value: 94.65599999999999
    - type: precision_at_1
      value: 91.855
    - type: precision_at_10
      value: 9.774
    - type: precision_at_100
      value: 0.9950000000000001
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.932
    - type: precision_at_3
      value: 31.976
    - type: precision_at_5
      value: 19.367
    - type: recall_at_1
      value: 91.855
    - type: recall_at_10
      value: 97.738
    - type: recall_at_100
      value: 99.548
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 98.643
    - type: recall_at_3
      value: 95.928
    - type: recall_at_5
      value: 96.833
    task:
      type: Retrieval
  - dataset:
      config: default
      name: MTEB CodeTransOceanDL (default)
      revision: 281562cb8a1265ab5c0824bfa6ddcd9b0a15618f
      split: test
      type: CoIR-Retrieval/codetrans-dl
    metrics:
    - type: main_score
      value: 38.566
    - type: map_at_1
      value: 8.889
    - type: map_at_10
      value: 24.311
    - type: map_at_100
      value: 25.413999999999998
    - type: map_at_1000
      value: 25.413999999999998
    - type: map_at_20
      value: 25.413999999999998
    - type: map_at_3
      value: 14.352
    - type: map_at_5
      value: 18.296
    - type: mrr_at_1
      value: 7.777777777777778
    - type: mrr_at_10
      value: 23.692460317460355
    - type: mrr_at_100
      value: 24.94246031746036
    - type: mrr_at_1000
      value: 24.94246031746036
    - type: mrr_at_20
      value: 24.94246031746036
    - type: mrr_at_3
      value: 13.611111111111104
    - type: mrr_at_5
      value: 18.13888888888888
    - type: nauc_map_at_1000_diff1
      value: -11.995791794456052
    - type: nauc_map_at_1000_max
      value: -37.4290224077524
    - type: nauc_map_at_1000_std
      value: -8.685310117692485
    - type: nauc_map_at_100_diff1
      value: -11.995791794456052
    - type: nauc_map_at_100_max
      value: -37.4290224077524
    - type: nauc_map_at_100_std
      value: -8.685310117692485
    - type: nauc_map_at_10_diff1
      value: -12.904069625259378
    - type: nauc_map_at_10_max
      value: -36.469676015537445
    - type: nauc_map_at_10_std
      value: -8.18556060930571
    - type: nauc_map_at_1_diff1
      value: -22.632143815401133
    - type: nauc_map_at_1_max
      value: -38.05674805473572
    - type: nauc_map_at_1_std
      value: -5.114703514891332
    - type: nauc_map_at_20_diff1
      value: -11.995791794456052
    - type: nauc_map_at_20_max
      value: -37.4290224077524
    - type: nauc_map_at_20_std
      value: -8.685310117692485
    - type: nauc_map_at_3_diff1
      value: -9.430569631049272
    - type: nauc_map_at_3_max
      value: -40.31306127773153
    - type: nauc_map_at_3_std
      value: -9.279256680145705
    - type: nauc_map_at_5_diff1
      value: -11.17196963640011
    - type: nauc_map_at_5_max
      value: -36.71021151035073
    - type: nauc_map_at_5_std
      value: -10.227972134518213
    - type: nauc_mrr_at_1000_diff1
      value: 7.811397652262392
    - type: nauc_mrr_at_1000_max
      value: -35.827281345154454
    - type: nauc_mrr_at_1000_std
      value: -10.625492656056048
    - type: nauc_mrr_at_100_diff1
      value: 7.811397652262392
    - type: nauc_mrr_at_100_max
      value: -35.827281345154454
    - type: nauc_mrr_at_100_std
      value: -10.625492656056048
    - type: nauc_mrr_at_10_diff1
      value: 7.0457873475325155
    - type: nauc_mrr_at_10_max
      value: -34.091630838685006
    - type: nauc_mrr_at_10_std
      value: -10.015877703911025
    - type: nauc_mrr_at_1_diff1
      value: 6.72505653723791
    - type: nauc_mrr_at_1_max
      value: -39.05860707577908
    - type: nauc_mrr_at_1_std
      value: -12.350032580781182
    - type: nauc_mrr_at_20_diff1
      value: 7.811397652262392
    - type: nauc_mrr_at_20_max
      value: -35.827281345154454
    - type: nauc_mrr_at_20_std
      value: -10.625492656056048
    - type: nauc_mrr_at_3_diff1
      value: 9.229424174654557
    - type: nauc_mrr_at_3_max
      value: -39.823960095369024
    - type: nauc_mrr_at_3_std
      value: -12.365662926663418
    - type: nauc_mrr_at_5_diff1
      value: 7.455314880931567
    - type: nauc_mrr_at_5_max
      value: -34.4606595687379
    - type: nauc_mrr_at_5_std
      value: -9.748438671046124
    - type: nauc_ndcg_at_1000_diff1
      value: -11.393187924360975
    - type: nauc_ndcg_at_1000_max
      value: -37.14382596573424
    - type: nauc_ndcg_at_1000_std
      value: -8.714465550011122
    - type: nauc_ndcg_at_100_diff1
      value: -11.393187924360975
    - type: nauc_ndcg_at_100_max
      value: -37.14382596573424
    - type: nauc_ndcg_at_100_std
      value: -8.714465550011122
    - type: nauc_ndcg_at_10_diff1
      value: -14.50305018622005
    - type: nauc_ndcg_at_10_max
      value: -34.183276513019024
    - type: nauc_ndcg_at_10_std
      value: -6.91808119392616
    - type: nauc_ndcg_at_1_diff1
      value: -22.632143815401133
    - type: nauc_ndcg_at_1_max
      value: -38.05674805473572
    - type: nauc_ndcg_at_1_std
      value: -5.114703514891332
    - type: nauc_ndcg_at_20_diff1
      value: -11.393187924360975
    - type: nauc_ndcg_at_20_max
      value: -37.14382596573424
    - type: nauc_ndcg_at_20_std
      value: -8.714465550011122
    - type: nauc_ndcg_at_3_diff1
      value: -6.28820826432392
    - type: nauc_ndcg_at_3_max
      value: -40.515698550813426
    - type: nauc_ndcg_at_3_std
      value: -9.763678652070176
    - type: nauc_ndcg_at_5_diff1
      value: -9.784255160012018
    - type: nauc_ndcg_at_5_max
      value: -35.547401094483774
    - type: nauc_ndcg_at_5_std
      value: -11.253198818827153
    - type: nauc_precision_at_1000_diff1
      value: 100.0
    - type: nauc_precision_at_1000_max
      value: 100.0
    - type: nauc_precision_at_1000_std
      value: 100.0
    - type: nauc_precision_at_100_diff1
      value: 100.0
    - type: nauc_precision_at_100_max
      value: 100.0
    - type: nauc_precision_at_100_std
      value: 100.0
    - type: nauc_precision_at_10_diff1
      value: -28.569339405841387
    - type: nauc_precision_at_10_max
      value: -21.317845652446533
    - type: nauc_precision_at_10_std
      value: 1.6546191451133108
    - type: nauc_precision_at_1_diff1
      value: -22.632143815401133
    - type: nauc_precision_at_1_max
      value: -38.05674805473572
    - type: nauc_precision_at_1_std
      value: -5.114703514891332
    - type: nauc_precision_at_20_diff1
      value: .nan
    - type: nauc_precision_at_20_max
      value: .nan
    - type: nauc_precision_at_20_std
      value: .nan
    - type: nauc_precision_at_3_diff1
      value: 0.21101534115564527
    - type: nauc_precision_at_3_max
      value: -40.8638015094404
    - type: nauc_precision_at_3_std
      value: -10.657976464982523
    - type: nauc_precision_at_5_diff1
      value: -7.729717358965525
    - type: nauc_precision_at_5_max
      value: -33.27683373383679
    - type: nauc_precision_at_5_std
      value: -13.203741846893195
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: .nan
    - type: nauc_recall_at_100_max
      value: .nan
    - type: nauc_recall_at_100_std
      value: .nan
    - type: nauc_recall_at_10_diff1
      value: -28.569339405841294
    - type: nauc_recall_at_10_max
      value: -21.317845652446415
    - type: nauc_recall_at_10_std
      value: 1.6546191451134153
    - type: nauc_recall_at_1_diff1
      value: -22.632143815401133
    - type: nauc_recall_at_1_max
      value: -38.05674805473572
    - type: nauc_recall_at_1_std
      value: -5.114703514891332
    - type: nauc_recall_at_20_diff1
      value: .nan
    - type: nauc_recall_at_20_max
      value: .nan
    - type: nauc_recall_at_20_std
      value: .nan
    - type: nauc_recall_at_3_diff1
      value: 0.2110153411556453
    - type: nauc_recall_at_3_max
      value: -40.863801509440385
    - type: nauc_recall_at_3_std
      value: -10.65797646498251
    - type: nauc_recall_at_5_diff1
      value: -7.729717358965576
    - type: nauc_recall_at_5_max
      value: -33.276833733836845
    - type: nauc_recall_at_5_std
      value: -13.203741846893251
    - type: ndcg_at_1
      value: 8.889
    - type: ndcg_at_10
      value: 38.566
    - type: ndcg_at_100
      value: 42.063
    - type: ndcg_at_1000
      value: 42.063
    - type: ndcg_at_20
      value: 42.063
    - type: ndcg_at_3
      value: 16.488
    - type: ndcg_at_5
      value: 23.589
    - type: precision_at_1
      value: 8.889
    - type: precision_at_10
      value: 8.722000000000001
    - type: precision_at_100
      value: 1.0
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 5.0
    - type: precision_at_3
      value: 7.593
    - type: precision_at_5
      value: 8.0
    - type: recall_at_1
      value: 8.889
    - type: recall_at_10
      value: 87.222
    - type: recall_at_100
      value: 100.0
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 100.0
    - type: recall_at_3
      value: 22.778000000000002
    - type: recall_at_5
      value: 40.0
    task:
      type: Retrieval
  - dataset:
      config: default
      name: MTEB CosQA (default)
      revision: bc5efb7e9d437246ce393ed19d772e08e4a79535
      split: test
      type: CoIR-Retrieval/cosqa
    metrics:
    - type: main_score
      value: 34.452
    - type: map_at_1
      value: 15.2
    - type: map_at_10
      value: 26.473000000000003
    - type: map_at_100
      value: 28.01
    - type: map_at_1000
      value: 28.038
    - type: map_at_20
      value: 27.48
    - type: map_at_3
      value: 20.833
    - type: map_at_5
      value: 24.153
    - type: mrr_at_1
      value: 19.2
    - type: mrr_at_10
      value: 28.281111111111105
    - type: mrr_at_100
      value: 29.8632158616367
    - type: mrr_at_1000
      value: 29.891199207757197
    - type: mrr_at_20
      value: 29.27876414692977
    - type: mrr_at_3
      value: 23.966666666666658
    - type: mrr_at_5
      value: 25.636666666666663
    - type: nauc_map_at_1000_diff1
      value: 24.10146846046148
    - type: nauc_map_at_1000_max
      value: 10.598410702361434
    - type: nauc_map_at_1000_std
      value: -20.144523237787222
    - type: nauc_map_at_100_diff1
      value: 24.053508672795086
    - type: nauc_map_at_100_max
      value: 10.62060173891184
    - type: nauc_map_at_100_std
      value: -20.123151261597684
    - type: nauc_map_at_10_diff1
      value: 24.03401073979636
    - type: nauc_map_at_10_max
      value: 10.204901836438324
    - type: nauc_map_at_10_std
      value: -20.723915647527605
    - type: nauc_map_at_1_diff1
      value: 34.9707775098606
    - type: nauc_map_at_1_max
      value: 8.792695974322537
    - type: nauc_map_at_1_std
      value: -17.679606534340426
    - type: nauc_map_at_20_diff1
      value: 23.635390281247904
    - type: nauc_map_at_20_max
      value: 10.51091267854992
    - type: nauc_map_at_20_std
      value: -20.503982338903548
    - type: nauc_map_at_3_diff1
      value: 27.08078023869985
    - type: nauc_map_at_3_max
      value: 8.624902998628762
    - type: nauc_map_at_3_std
      value: -21.16290389423999
    - type: nauc_map_at_5_diff1
      value: 24.375227072550878
    - type: nauc_map_at_5_max
      value: 10.446847054213334
    - type: nauc_map_at_5_std
      value: -20.416673549725004
    - type: nauc_mrr_at_1000_diff1
      value: 19.78435935694305
    - type: nauc_mrr_at_1000_max
      value: 6.389276526733645
    - type: nauc_mrr_at_1000_std
      value: -24.87029202656466
    - type: nauc_mrr_at_100_diff1
      value: 19.741897116785875
    - type: nauc_mrr_at_100_max
      value: 6.399527259062092
    - type: nauc_mrr_at_100_std
      value: -24.86454789316169
    - type: nauc_mrr_at_10_diff1
      value: 19.82954896251409
    - type: nauc_mrr_at_10_max
      value: 5.841515376599231
    - type: nauc_mrr_at_10_std
      value: -25.585472957584926
    - type: nauc_mrr_at_1_diff1
      value: 22.61366365164493
    - type: nauc_mrr_at_1_max
      value: 5.888630966674663
    - type: nauc_mrr_at_1_std
      value: -23.551522185757364
    - type: nauc_mrr_at_20_diff1
      value: 19.610569618359854
    - type: nauc_mrr_at_20_max
      value: 6.388923736326994
    - type: nauc_mrr_at_20_std
      value: -25.022579250123666
    - type: nauc_mrr_at_3_diff1
      value: 21.689261589102053
    - type: nauc_mrr_at_3_max
      value: 5.433511684692397
    - type: nauc_mrr_at_3_std
      value: -25.22461843203615
    - type: nauc_mrr_at_5_diff1
      value: 20.769666984905864
    - type: nauc_mrr_at_5_max
      value: 5.103054542191035
    - type: nauc_mrr_at_5_std
      value: -25.62916796715137
    - type: nauc_ndcg_at_1000_diff1
      value: 21.724475676486286
    - type: nauc_ndcg_at_1000_max
      value: 11.92044108916809
    - type: nauc_ndcg_at_1000_std
      value: -19.560471367028224
    - type: nauc_ndcg_at_100_diff1
      value: 20.65779971541893
    - type: nauc_ndcg_at_100_max
      value: 12.772564966149327
    - type: nauc_ndcg_at_100_std
      value: -18.53040956973807
    - type: nauc_ndcg_at_10_diff1
      value: 19.715388368513594
    - type: nauc_ndcg_at_10_max
      value: 11.290056737017848
    - type: nauc_ndcg_at_10_std
      value: -21.48987713396045
    - type: nauc_ndcg_at_1_diff1
      value: 34.9707775098606
    - type: nauc_ndcg_at_1_max
      value: 8.792695974322537
    - type: nauc_ndcg_at_1_std
      value: -17.679606534340426
    - type: nauc_ndcg_at_20_diff1
      value: 18.26515065287157
    - type: nauc_ndcg_at_20_max
      value: 12.420581957680346
    - type: nauc_ndcg_at_20_std
      value: -20.560776758982286
    - type: nauc_ndcg_at_3_diff1
      value: 24.86486238625192
    - type: nauc_ndcg_at_3_max
      value: 8.51016243931613
    - type: nauc_ndcg_at_3_std
      value: -22.38799975521682
    - type: nauc_ndcg_at_5_diff1
      value: 20.38709195941383
    - type: nauc_ndcg_at_5_max
      value: 11.692332914550612
    - type: nauc_ndcg_at_5_std
      value: -21.095128458420735
    - type: nauc_precision_at_1000_diff1
      value: -2.427637721760636
    - type: nauc_precision_at_1000_max
      value: 93.46405228758199
    - type: nauc_precision_at_1000_std
      value: 56.139122315591536
    - type: nauc_precision_at_100_diff1
      value: -12.075404874593668
    - type: nauc_precision_at_100_max
      value: 46.41166811552213
    - type: nauc_precision_at_100_std
      value: 21.43662062526193
    - type: nauc_precision_at_10_diff1
      value: 7.927965224591165
    - type: nauc_precision_at_10_max
      value: 14.499053516090232
    - type: nauc_precision_at_10_std
      value: -23.215405338109257
    - type: nauc_precision_at_1_diff1
      value: 34.9707775098606
    - type: nauc_precision_at_1_max
      value: 8.792695974322537
    - type: nauc_precision_at_1_std
      value: -17.679606534340426
    - type: nauc_precision_at_20_diff1
      value: -2.2220305852593376
    - type: nauc_precision_at_20_max
      value: 20.688168333908145
    - type: nauc_precision_at_20_std
      value: -19.112501847927895
    - type: nauc_precision_at_3_diff1
      value: 19.523274018213513
    - type: nauc_precision_at_3_max
      value: 8.217018367841955
    - type: nauc_precision_at_3_std
      value: -25.40158982161363
    - type: nauc_precision_at_5_diff1
      value: 10.681426032017447
    - type: nauc_precision_at_5_max
      value: 14.961711488638585
    - type: nauc_precision_at_5_std
      value: -22.588397199647623
    - type: nauc_recall_at_1000_diff1
      value: -2.4276377217553624
    - type: nauc_recall_at_1000_max
      value: 93.46405228758147
    - type: nauc_recall_at_1000_std
      value: 56.13912231559195
    - type: nauc_recall_at_100_diff1
      value: -12.075404874593449
    - type: nauc_recall_at_100_max
      value: 46.41166811552219
    - type: nauc_recall_at_100_std
      value: 21.436620625261668
    - type: nauc_recall_at_10_diff1
      value: 7.927965224591155
    - type: nauc_recall_at_10_max
      value: 14.499053516090227
    - type: nauc_recall_at_10_std
      value: -23.21540533810927
    - type: nauc_recall_at_1_diff1
      value: 34.9707775098606
    - type: nauc_recall_at_1_max
      value: 8.792695974322537
    - type: nauc_recall_at_1_std
      value: -17.679606534340426
    - type: nauc_recall_at_20_diff1
      value: -2.22203058525925
    - type: nauc_recall_at_20_max
      value: 20.688168333908262
    - type: nauc_recall_at_20_std
      value: -19.11250184792784
    - type: nauc_recall_at_3_diff1
      value: 19.523274018213503
    - type: nauc_recall_at_3_max
      value: 8.217018367841955
    - type: nauc_recall_at_3_std
      value: -25.40158982161363
    - type: nauc_recall_at_5_diff1
      value: 10.681426032017423
    - type: nauc_recall_at_5_max
      value: 14.961711488638551
    - type: nauc_recall_at_5_std
      value: -22.58839719964763
    - type: ndcg_at_1
      value: 15.2
    - type: ndcg_at_10
      value: 34.452
    - type: ndcg_at_100
      value: 41.723
    - type: ndcg_at_1000
      value: 42.421
    - type: ndcg_at_20
      value: 38.002
    - type: ndcg_at_3
      value: 22.864
    - type: ndcg_at_5
      value: 28.904999999999998
    - type: precision_at_1
      value: 15.2
    - type: precision_at_10
      value: 6.0600000000000005
    - type: precision_at_100
      value: 0.942
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 3.7199999999999998
    - type: precision_at_3
      value: 9.6
    - type: precision_at_5
      value: 8.72
    - type: recall_at_1
      value: 15.2
    - type: recall_at_10
      value: 60.6
    - type: recall_at_100
      value: 94.19999999999999
    - type: recall_at_1000
      value: 99.6
    - type: recall_at_20
      value: 74.4
    - type: recall_at_3
      value: 28.799999999999997
    - type: recall_at_5
      value: 43.6
    task:
      type: Retrieval
  - dataset:
      config: default
      name: MTEB StackOverflowQA (default)
      revision: db8f169f3894c14a00251061f957b2063eef2bd5
      split: test
      type: CoIR-Retrieval/stackoverflow-qa
    metrics:
    - type: main_score
      value: 97.16799999999999
    - type: map_at_1
      value: 94.985
    - type: map_at_10
      value: 96.533
    - type: map_at_100
      value: 96.568
    - type: map_at_1000
      value: 96.569
    - type: map_at_20
      value: 96.554
    - type: map_at_3
      value: 96.272
    - type: map_at_5
      value: 96.48
    - type: mrr_at_1
      value: 94.98495486459379
    - type: mrr_at_10
      value: 96.53325054528665
    - type: mrr_at_100
      value: 96.56849512537934
    - type: mrr_at_1000
      value: 96.56899560792449
    - type: mrr_at_20
      value: 96.55354401474338
    - type: mrr_at_3
      value: 96.27214978268137
    - type: mrr_at_5
      value: 96.48027415580073
    - type: nauc_map_at_1000_diff1
      value: 96.7701081739238
    - type: nauc_map_at_1000_max
      value: 89.92147554128405
    - type: nauc_map_at_1000_std
      value: -11.367860416596368
    - type: nauc_map_at_100_diff1
      value: 96.7701266461378
    - type: nauc_map_at_100_max
      value: 89.92249288169609
    - type: nauc_map_at_100_std
      value: -11.365088267343506
    - type: nauc_map_at_10_diff1
      value: 96.80296303411313
    - type: nauc_map_at_10_max
      value: 89.97144795621024
    - type: nauc_map_at_10_std
      value: -11.225883175842885
    - type: nauc_map_at_1_diff1
      value: 96.33579234749389
    - type: nauc_map_at_1_max
      value: 88.14831631499773
    - type: nauc_map_at_1_std
      value: -11.043584429943797
    - type: nauc_map_at_20_diff1
      value: 96.7841381971511
    - type: nauc_map_at_20_max
      value: 89.96621023207143
    - type: nauc_map_at_20_std
      value: -11.236908272386724
    - type: nauc_map_at_3_diff1
      value: 96.77341739930384
    - type: nauc_map_at_3_max
      value: 90.18226749615211
    - type: nauc_map_at_3_std
      value: -12.405699503933928
    - type: nauc_map_at_5_diff1
      value: 96.79677069369804
    - type: nauc_map_at_5_max
      value: 90.04406323888772
    - type: nauc_map_at_5_std
      value: -11.09579820828818
    - type: nauc_mrr_at_1000_diff1
      value: 96.7701081739238
    - type: nauc_mrr_at_1000_max
      value: 89.92147554128405
    - type: nauc_mrr_at_1000_std
      value: -11.367860416596368
    - type: nauc_mrr_at_100_diff1
      value: 96.7701266461378
    - type: nauc_mrr_at_100_max
      value: 89.92249288169609
    - type: nauc_mrr_at_100_std
      value: -11.365088267343506
    - type: nauc_mrr_at_10_diff1
      value: 96.80296303411313
    - type: nauc_mrr_at_10_max
      value: 89.97144795621024
    - type: nauc_mrr_at_10_std
      value: -11.225883175842885
    - type: nauc_mrr_at_1_diff1
      value: 96.33579234749389
    - type: nauc_mrr_at_1_max
      value: 88.14831631499773
    - type: nauc_mrr_at_1_std
      value: -11.043584429943797
    - type: nauc_mrr_at_20_diff1
      value: 96.7841381971511
    - type: nauc_mrr_at_20_max
      value: 89.96621023207143
    - type: nauc_mrr_at_20_std
      value: -11.236908272386724
    - type: nauc_mrr_at_3_diff1
      value: 96.77341739930384
    - type: nauc_mrr_at_3_max
      value: 90.18226749615211
    - type: nauc_mrr_at_3_std
      value: -12.405699503933928
    - type: nauc_mrr_at_5_diff1
      value: 96.79677069369804
    - type: nauc_mrr_at_5_max
      value: 90.04406323888772
    - type: nauc_mrr_at_5_std
      value: -11.09579820828818
    - type: nauc_ndcg_at_1000_diff1
      value: 96.83153135375731
    - type: nauc_ndcg_at_1000_max
      value: 90.17747818029089
    - type: nauc_ndcg_at_1000_std
      value: -11.09225768112506
    - type: nauc_ndcg_at_100_diff1
      value: 96.82500676553634
    - type: nauc_ndcg_at_100_max
      value: 90.21640577830735
    - type: nauc_ndcg_at_100_std
      value: -11.051187241816189
    - type: nauc_ndcg_at_10_diff1
      value: 97.01040984737975
    - type: nauc_ndcg_at_10_max
      value: 90.53590661634338
    - type: nauc_ndcg_at_10_std
      value: -10.181364845409425
    - type: nauc_ndcg_at_1_diff1
      value: 96.33579234749389
    - type: nauc_ndcg_at_1_max
      value: 88.14831631499773
    - type: nauc_ndcg_at_1_std
      value: -11.043584429943797
    - type: nauc_ndcg_at_20_diff1
      value: 96.92876646822131
    - type: nauc_ndcg_at_20_max
      value: 90.53613648406619
    - type: nauc_ndcg_at_20_std
      value: -10.211015921117655
    - type: nauc_ndcg_at_3_diff1
      value: 96.92301963415002
    - type: nauc_ndcg_at_3_max
      value: 90.96341737957319
    - type: nauc_ndcg_at_3_std
      value: -12.397871226017621
    - type: nauc_ndcg_at_5_diff1
      value: 96.99223706029623
    - type: nauc_ndcg_at_5_max
      value: 90.7297142361567
    - type: nauc_ndcg_at_5_std
      value: -9.78987977360544
    - type: nauc_precision_at_1000_diff1
      value: .nan
    - type: nauc_precision_at_1000_max
      value: .nan
    - type: nauc_precision_at_1000_std
      value: .nan
    - type: nauc_precision_at_100_diff1
      value: 95.64367251079331
    - type: nauc_precision_at_100_max
      value: 95.64367251079331
    - type: nauc_precision_at_100_std
      value: -7.9737174815413105
    - type: nauc_precision_at_10_diff1
      value: 99.27394541846478
    - type: nauc_precision_at_10_max
      value: 96.07116729984303
    - type: nauc_precision_at_10_std
      value: 2.384368710766077
    - type: nauc_precision_at_1_diff1
      value: 96.33579234749389
    - type: nauc_precision_at_1_max
      value: 88.14831631499773
    - type: nauc_precision_at_1_std
      value: -11.043584429943797
    - type: nauc_precision_at_20_diff1
      value: 98.91091812769709
    - type: nauc_precision_at_20_max
      value: 98.91091812769709
    - type: nauc_precision_at_20_std
      value: 8.043758582511515
    - type: nauc_precision_at_3_diff1
      value: 97.6238213695205
    - type: nauc_precision_at_3_max
      value: 94.67950559666251
    - type: nauc_precision_at_3_std
      value: -12.047680267059418
    - type: nauc_precision_at_5_diff1
      value: 98.49204048450248
    - type: nauc_precision_at_5_max
      value: 95.77207938439504
    - type: nauc_precision_at_5_std
      value: 1.5409353000901833
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: 95.64367251078416
    - type: nauc_recall_at_100_max
      value: 95.64367251078416
    - type: nauc_recall_at_100_std
      value: -7.973717481540185
    - type: nauc_recall_at_10_diff1
      value: 99.27394541846407
    - type: nauc_recall_at_10_max
      value: 96.07116729984456
    - type: nauc_recall_at_10_std
      value: 2.3843687107667484
    - type: nauc_recall_at_1_diff1
      value: 96.33579234749389
    - type: nauc_recall_at_1_max
      value: 88.14831631499773
    - type: nauc_recall_at_1_std
      value: -11.043584429943797
    - type: nauc_recall_at_20_diff1
      value: 98.91091812769824
    - type: nauc_recall_at_20_max
      value: 98.91091812769824
    - type: nauc_recall_at_20_std
      value: 8.043758582515963
    - type: nauc_recall_at_3_diff1
      value: 97.62382136952051
    - type: nauc_recall_at_3_max
      value: 94.67950559666313
    - type: nauc_recall_at_3_std
      value: -12.047680267059349
    - type: nauc_recall_at_5_diff1
      value: 98.49204048450328
    - type: nauc_recall_at_5_max
      value: 95.77207938439567
    - type: nauc_recall_at_5_std
      value: 1.54093530009189
    - type: ndcg_at_1
      value: 94.985
    - type: ndcg_at_10
      value: 97.16799999999999
    - type: ndcg_at_100
      value: 97.333
    - type: ndcg_at_1000
      value: 97.351
    - type: ndcg_at_20
      value: 97.243
    - type: ndcg_at_3
      value: 96.665
    - type: ndcg_at_5
      value: 97.038
    - type: precision_at_1
      value: 94.985
    - type: precision_at_10
      value: 9.91
    - type: precision_at_100
      value: 0.9979999999999999
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.97
    - type: precision_at_3
      value: 32.598
    - type: precision_at_5
      value: 19.739
    - type: recall_at_1
      value: 94.985
    - type: recall_at_10
      value: 99.09700000000001
    - type: recall_at_100
      value: 99.85000000000001
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 99.398
    - type: recall_at_3
      value: 97.79299999999999
    - type: recall_at_5
      value: 98.696
    task:
      type: Retrieval
  - dataset:
      config: default
      name: MTEB SyntheticText2SQL (default)
      revision: 686b87296c3a0191b5d9415a00526c62db9fce09
      split: test
      type: CoIR-Retrieval/synthetic-text2sql
    metrics:
    - type: main_score
      value: 62.871
    - type: map_at_1
      value: 5.931
    - type: map_at_10
      value: 50.172000000000004
    - type: map_at_100
      value: 50.221000000000004
    - type: map_at_1000
      value: 50.221000000000004
    - type: map_at_20
      value: 50.217
    - type: map_at_3
      value: 49.126
    - type: map_at_5
      value: 49.966
    - type: mrr_at_1
      value: 46.84669287301316
    - type: mrr_at_10
      value: 71.28829558371541
    - type: mrr_at_100
      value: 71.33501465192626
    - type: mrr_at_1000
      value: 71.33501465192626
    - type: mrr_at_20
      value: 71.33145566423184
    - type: mrr_at_3
      value: 70.51501167891556
    - type: mrr_at_5
      value: 71.10978180368052
    - type: nauc_map_at_1000_diff1
      value: 16.85059525629914
    - type: nauc_map_at_1000_max
      value: 7.2095269963540725
    - type: nauc_map_at_1000_std
      value: -31.394151564475674
    - type: nauc_map_at_100_diff1
      value: 16.85059525629914
    - type: nauc_map_at_100_max
      value: 7.2095269963540725
    - type: nauc_map_at_100_std
      value: -31.394151564475674
    - type: nauc_map_at_10_diff1
      value: 16.005030500086086
    - type: nauc_map_at_10_max
      value: 7.264507684893849
    - type: nauc_map_at_10_std
      value: -31.715470923775076
    - type: nauc_map_at_1_diff1
      value: 88.69496772604496
    - type: nauc_map_at_1_max
      value: 2.919830321081601
    - type: nauc_map_at_1_std
      value: -12.316988109913101
    - type: nauc_map_at_20_diff1
      value: 16.782304707510036
    - type: nauc_map_at_20_max
      value: 7.245475061050962
    - type: nauc_map_at_20_std
      value: -31.399548485799105
    - type: nauc_map_at_3_diff1
      value: 0.7297119245486146
    - type: nauc_map_at_3_max
      value: 8.421968652977062
    - type: nauc_map_at_3_std
      value: -36.37536179543213
    - type: nauc_map_at_5_diff1
      value: 12.348535589464996
    - type: nauc_map_at_5_max
      value: 7.675248350287251
    - type: nauc_map_at_5_std
      value: -33.03849075726066
    - type: nauc_mrr_at_1000_diff1
      value: -19.145171066076582
    - type: nauc_mrr_at_1000_max
      value: 3.456240689954698
    - type: nauc_mrr_at_1000_std
      value: -16.911698657686483
    - type: nauc_mrr_at_100_diff1
      value: -19.145171066076582
    - type: nauc_mrr_at_100_max
      value: 3.456240689954698
    - type: nauc_mrr_at_100_std
      value: -16.911698657686483
    - type: nauc_mrr_at_10_diff1
      value: -19.383555832224395
    - type: nauc_mrr_at_10_max
      value: 3.4956712263663245
    - type: nauc_mrr_at_10_std
      value: -17.088724852125562
    - type: nauc_mrr_at_1_diff1
      value: -7.248689506871619
    - type: nauc_mrr_at_1_max
      value: 1.6279165510593367
    - type: nauc_mrr_at_1_std
      value: -10.76401320944665
    - type: nauc_mrr_at_20_diff1
      value: -19.166540939871457
    - type: nauc_mrr_at_20_max
      value: 3.471598334930003
    - type: nauc_mrr_at_20_std
      value: -16.91659546762138
    - type: nauc_mrr_at_3_diff1
      value: -23.156601373830686
    - type: nauc_mrr_at_3_max
      value: 4.041242636869845
    - type: nauc_mrr_at_3_std
      value: -19.05754313395974
    - type: nauc_mrr_at_5_diff1
      value: -20.312683178010765
    - type: nauc_mrr_at_5_max
      value: 3.617044817759521
    - type: nauc_mrr_at_5_std
      value: -17.611736247571123
    - type: nauc_ndcg_at_1000_diff1
      value: 12.200580013571008
    - type: nauc_ndcg_at_1000_max
      value: 7.5728136486801105
    - type: nauc_ndcg_at_1000_std
      value: -32.773584410427645
    - type: nauc_ndcg_at_100_diff1
      value: 12.200580013571008
    - type: nauc_ndcg_at_100_max
      value: 7.5728136486801105
    - type: nauc_ndcg_at_100_std
      value: -32.773584410427645
    - type: nauc_ndcg_at_10_diff1
      value: 8.346563719126793
    - type: nauc_ndcg_at_10_max
      value: 7.984058762338721
    - type: nauc_ndcg_at_10_std
      value: -34.20079429928775
    - type: nauc_ndcg_at_1_diff1
      value: 88.69496772604496
    - type: nauc_ndcg_at_1_max
      value: 2.919830321081601
    - type: nauc_ndcg_at_1_std
      value: -12.316988109913101
    - type: nauc_ndcg_at_20_diff1
      value: 11.618780757172203
    - type: nauc_ndcg_at_20_max
      value: 7.879670436948672
    - type: nauc_ndcg_at_20_std
      value: -32.832497615096266
    - type: nauc_ndcg_at_3_diff1
      value: -21.635858761258074
    - type: nauc_ndcg_at_3_max
      value: 10.142935236930697
    - type: nauc_ndcg_at_3_std
      value: -43.48822107044396
    - type: nauc_ndcg_at_5_diff1
      value: -1.4973636275297837
    - type: nauc_ndcg_at_5_max
      value: 8.981298991414445
    - type: nauc_ndcg_at_5_std
      value: -37.87476407099174
    - type: nauc_precision_at_1000_diff1
      value: .nan
    - type: nauc_precision_at_1000_max
      value: .nan
    - type: nauc_precision_at_1000_std
      value: .nan
    - type: nauc_precision_at_100_diff1
      value: .nan
    - type: nauc_precision_at_100_max
      value: .nan
    - type: nauc_precision_at_100_std
      value: .nan
    - type: nauc_precision_at_10_diff1
      value: -121.58511191301635
    - type: nauc_precision_at_10_max
      value: 25.248667404560425
    - type: nauc_precision_at_10_std
      value: -81.22406477400408
    - type: nauc_precision_at_1_diff1
      value: 88.69496772604496
    - type: nauc_precision_at_1_max
      value: 2.919830321081601
    - type: nauc_precision_at_1_std
      value: -12.316988109913101
    - type: nauc_precision_at_20_diff1
      value: -140.40859643368495
    - type: nauc_precision_at_20_max
      value: 87.46836627148437
    - type: nauc_precision_at_20_std
      value: -50.273460814697444
    - type: nauc_precision_at_3_diff1
      value: -116.53772258202089
    - type: nauc_precision_at_3_max
      value: 17.67074016899205
    - type: nauc_precision_at_3_std
      value: -74.36661207778327
    - type: nauc_precision_at_5_diff1
      value: -123.86337036708947
    - type: nauc_precision_at_5_max
      value: 21.909069776986158
    - type: nauc_precision_at_5_std
      value: -83.69114367867371
    - type: nauc_recall_at_1000_diff1
      value: .nan
    - type: nauc_recall_at_1000_max
      value: .nan
    - type: nauc_recall_at_1000_std
      value: .nan
    - type: nauc_recall_at_100_diff1
      value: .nan
    - type: nauc_recall_at_100_max
      value: .nan
    - type: nauc_recall_at_100_std
      value: .nan
    - type: nauc_recall_at_10_diff1
      value: -121.58511191302262
    - type: nauc_recall_at_10_max
      value: 25.24866740455998
    - type: nauc_recall_at_10_std
      value: -81.22406477400794
    - type: nauc_recall_at_1_diff1
      value: 88.69496772604496
    - type: nauc_recall_at_1_max
      value: 2.919830321081601
    - type: nauc_recall_at_1_std
      value: -12.316988109913101
    - type: nauc_recall_at_20_diff1
      value: -140.40859643373693
    - type: nauc_recall_at_20_max
      value: 87.46836627148096
    - type: nauc_recall_at_20_std
      value: -50.27346081473573
    - type: nauc_recall_at_3_diff1
      value: -116.53772258202
    - type: nauc_recall_at_3_max
      value: 17.670740168992516
    - type: nauc_recall_at_3_std
      value: -74.3666120777825
    - type: nauc_recall_at_5_diff1
      value: -123.86337036708981
    - type: nauc_recall_at_5_max
      value: 21.909069776986765
    - type: nauc_recall_at_5_std
      value: -83.69114367867463
    - type: ndcg_at_1
      value: 5.931
    - type: ndcg_at_10
      value: 62.871
    - type: ndcg_at_100
      value: 63.05
    - type: ndcg_at_1000
      value: 63.05
    - type: ndcg_at_20
      value: 63.027
    - type: ndcg_at_3
      value: 60.89
    - type: ndcg_at_5
      value: 62.382000000000005
    - type: precision_at_1
      value: 5.931
    - type: precision_at_10
      value: 9.927999999999999
    - type: precision_at_100
      value: 1.0
    - type: precision_at_1000
      value: 0.1
    - type: precision_at_20
      value: 4.994
    - type: precision_at_3
      value: 31.408
    - type: precision_at_5
      value: 19.559
    - type: recall_at_1
      value: 5.931
    - type: recall_at_10
      value: 99.28200000000001
    - type: recall_at_100
      value: 100.0
    - type: recall_at_1000
      value: 100.0
    - type: recall_at_20
      value: 99.88
    - type: recall_at_3
      value: 94.223
    - type: recall_at_5
      value: 97.795
    task:
      type: Retrieval
tags:
- mteb
---
# voyage-code-3

This repo contains the tokenizer of the `voyage-code-3` embedding model. `voyage-code-3` is optimized for code retrieval, outperforming OpenAI-v3-large and CodeSage-large by an average of 13.80% and 16.81% on a suite of 238 code retrieval datasets, respectively. By supporting smaller dimensions with Matryoshka learning and quantized formats like int8 and binary, voyage-code-3 can also dramatically reduce storage and search costs with minimal impact on retrieval quality. Please refer to [our blogpost](https://blog.voyageai.com/2024/12/04/voyage-code-3) for more details about this model.

You can use `voyage-code-3` via the [Voyage API](https://docs.voyageai.com/docs/embeddings), [AWS SageMaker](https://aws.amazon.com/marketplace/pp/prodview-d5nri3kbddsrw) or on-prem deplotment.

## Voyage API
The following code snippet shows the usage of the Voyage API. Install the Voyage AI Python SDK via:

```
pip install -U voyageai
```

Voyage AI utilizes API keys to monitor usage and manage permissions. To obtain your key, please sign in with your Voyage AI account and click the "Create new API key" button in the dashboard. We recommend setting the API key as an environment variable. For example, in MacOS or Linux, type the following command in the terminal, replacing <your secret key> with your actual API key:

```python
import voyageai

vo = voyageai.Client()
# This will automatically use the environment variable VOYAGE_API_KEY.
# Alternatively, you can use vo = voyageai.Client(api_key="<your secret key>")

result = vo.embed(
    texts=["hello world"], 
    model="voyage-code-3", 
    input_type="document",
    output_dimension=2048,
    output_dtype="float"
)
```
## AWS SageMaker

The embedding model can be privately deployed in your AWS Cloud using our [AWS SageMaker marketplace offering](https://aws.amazon.com/marketplace/pp/prodview-d5nri3kbddsrw). It runs privately in your VPC.

## On-prem Deployment

Want to run `voyage-code-3` on your own hardware? Feel free to reach out to contact@voyageai.com to learn more.