Create perplexity.txt

perplexity.txt

@@ -0,0 +1,115 @@
+
+
+
+
+
+
+
+
+
+
+
+
+IQ4_XS
+
+```
+./perplexity -f /home/user/Downloads/wikitext-2-raw/wiki.test.raw -m ../../IQ4_XS_120B 
+main: build = 2709 (40f74e4d)
+main: built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu
+main: seed  = 1713741077
+llama_model_loader: loaded meta data with 22 key-value pairs and 1263 tensors from ../../IQ4_XS_120B (version GGUF V3 (latest))
+llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
+llama_model_loader: - kv   0:                       general.architecture str              = llama
+llama_model_loader: - kv   1:                               general.name str              = New Volume
+llama_model_loader: - kv   2:                           llama.vocab_size u32              = 128256
+llama_model_loader: - kv   3:                       llama.context_length u32              = 8192
+llama_model_loader: - kv   4:                     llama.embedding_length u32              = 8192
+llama_model_loader: - kv   5:                          llama.block_count u32              = 140
+llama_model_loader: - kv   6:                  llama.feed_forward_length u32              = 28672
+llama_model_loader: - kv   7:                 llama.rope.dimension_count u32              = 128
+llama_model_loader: - kv   8:                 llama.attention.head_count u32              = 64
+llama_model_loader: - kv   9:              llama.attention.head_count_kv u32              = 8
+llama_model_loader: - kv  10:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
+llama_model_loader: - kv  11:                       llama.rope.freq_base f32              = 500000.000000
+llama_model_loader: - kv  12:                          general.file_type u32              = 30
+llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = gpt2
+llama_model_loader: - kv  14:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
+llama_model_loader: - kv  15:                      tokenizer.ggml.scores arr[f32,128256]  = [0.000000, 0.000000, 0.000000, 0.0000...
+llama_model_loader: - kv  16:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
+llama_model_loader: - kv  17:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
+llama_model_loader: - kv  18:                tokenizer.ggml.bos_token_id u32              = 128000
+llama_model_loader: - kv  19:                tokenizer.ggml.eos_token_id u32              = 128001
+llama_model_loader: - kv  20:                    tokenizer.chat_template str              = {% set loop_messages = messages %}{% ...
+llama_model_loader: - kv  21:               general.quantization_version u32              = 2
+llama_model_loader: - type  f32:  281 tensors
+llama_model_loader: - type q5_K:  157 tensors
+llama_model_loader: - type q6_K:    1 tensors
+llama_model_loader: - type iq4_xs:  824 tensors
+llm_load_vocab: special tokens definition check successful ( 256/128256 ).
+llm_load_print_meta: format           = GGUF V3 (latest)
+llm_load_print_meta: arch             = llama
+llm_load_print_meta: vocab type       = BPE
+llm_load_print_meta: n_vocab          = 128256
+llm_load_print_meta: n_merges         = 280147
+llm_load_print_meta: n_ctx_train      = 8192
+llm_load_print_meta: n_embd           = 8192
+llm_load_print_meta: n_head           = 64
+llm_load_print_meta: n_head_kv        = 8
+llm_load_print_meta: n_layer          = 140
+llm_load_print_meta: n_rot            = 128
+llm_load_print_meta: n_embd_head_k    = 128
+llm_load_print_meta: n_embd_head_v    = 128
+llm_load_print_meta: n_gqa            = 8
+llm_load_print_meta: n_embd_k_gqa     = 1024
+llm_load_print_meta: n_embd_v_gqa     = 1024
+llm_load_print_meta: f_norm_eps       = 0.0e+00
+llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
+llm_load_print_meta: f_clamp_kqv      = 0.0e+00
+llm_load_print_meta: f_max_alibi_bias = 0.0e+00
+llm_load_print_meta: f_logit_scale    = 0.0e+00
+llm_load_print_meta: n_ff             = 28672
+llm_load_print_meta: n_expert         = 0
+llm_load_print_meta: n_expert_used    = 0
+llm_load_print_meta: causal attn      = 1
+llm_load_print_meta: pooling type     = 0
+llm_load_print_meta: rope type        = 0
+llm_load_print_meta: rope scaling     = linear
+llm_load_print_meta: freq_base_train  = 500000.0
+llm_load_print_meta: freq_scale_train = 1
+llm_load_print_meta: n_yarn_orig_ctx  = 8192
+llm_load_print_meta: rope_finetuned   = unknown
+llm_load_print_meta: ssm_d_conv       = 0
+llm_load_print_meta: ssm_d_inner      = 0
+llm_load_print_meta: ssm_d_state      = 0
+llm_load_print_meta: ssm_dt_rank      = 0
+llm_load_print_meta: model type       = ?B
+llm_load_print_meta: model ftype      = IQ4_XS - 4.25 bpw
+llm_load_print_meta: model params     = 121.89 B
+llm_load_print_meta: model size       = 61.35 GiB (4.32 BPW) 
+llm_load_print_meta: general.name     = New Volume
+llm_load_print_meta: BOS token        = 128000 '<|begin_of_text|>'
+llm_load_print_meta: EOS token        = 128001 '<|end_of_text|>'
+llm_load_print_meta: LF token         = 128 'Ä'
+llm_load_print_meta: EOT token        = 128009 '<|eot_id|>'
+llm_load_tensors: ggml ctx size =    0.64 MiB
+llm_load_tensors:        CPU buffer size = 62823.05 MiB
+...................................................................................................
+llama_new_context_with_model: n_ctx      = 2048
+llama_new_context_with_model: n_batch    = 2048
+llama_new_context_with_model: n_ubatch   = 512
+llama_new_context_with_model: freq_base  = 500000.0
+llama_new_context_with_model: freq_scale = 1
+llama_kv_cache_init:        CPU KV buffer size =  1120.00 MiB
+llama_new_context_with_model: KV self size  = 1120.00 MiB, K (f16):  560.00 MiB, V (f16):  560.00 MiB
+llama_new_context_with_model:        CPU  output buffer size =     1.96 MiB
+llama_new_context_with_model:        CPU compute buffer size =   324.01 MiB
+llama_new_context_with_model: graph nodes  = 4486
+llama_new_context_with_model: graph splits = 1
+
+system_info: n_threads = 16 / 32 | AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | 
+perplexity: tokenizing the input ..
+perplexity: tokenization took 1254.33 ms
+perplexity: calculating perplexity over 564 chunks, n_ctx=512, batch_size=2048, n_seq=4
+perplexity: 789.77 seconds per pass - ETA 30 hours 55.97 minutes
+[1]6.6257,[2]7.0895,[3]7.4198,[4]7.8431,[5]8.1498,[6]8.2868,[7]8.5416,[8]16.7631,[9]16.9070,[10]17.0177,[11]17.3416,[12]17.1153,[13]17.1495,[14]15.9925,[15]15.6819,[16]14.8677,[17]14.6194,[18]14.7188,[19]14.3299,[20]13.8694,[21]13.7140,[22]13.0065,[23]13.4134,[24]12.9553,[25]12.3752,[26]15.8985,[27]15.3589,[28]14.8173,[29]17.5358,[30]17.0630,[31]16.7671,[32]16.4692,[33]16.2825,[34]16.0765,[35]15.9255,[36]15.9421,[37]15.6712,[38]15.5823,[39]15.4130,[40]15.2759,[41]15.1863,[42]14.8463,[43]14.8449,[44]15.5276,[45]15.5749,[46]15.3805,[47]15.2628,[48]14.9983,[49]14.7855,[50]14.7951,[51]14.8063,[52]15.4632,[53]15.5870,[54]15.4298,[55]15.2675,[56]15.2484,[57]15.0987,[58]16.4450,[59]16.1559,[60]16.0842,[61]16.0550,[62]16.0266,[63]16.0539,[64]16.0921,[65]15.9508,[66]15.8068,[67]15.6202,[68]15.6122,[69]15.6199,[70]15.5773,[71]15.4804,[72]15.2780,[73]15.1460,[74]15.0568,[75]14.8374,[76]14.6901,[77]14.5159,[78]14.4655,[79]14.4193,[80]14.3641,[81]14.2662,[82]14.2472,[83]14.1861,[84]14.1679,[85]14.0669,[86]13.9520,[87]14.0137,[88]13.9075,[89]13.8810,[90]13.8321,[91]13.7511,[92]13.6789,
+```